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NATIONAL  BUREAU  OF  STANDARDS 


The  National  Bureau  of  Standards'  was  established  by  an  act  of  Congress  on  March  3,  1901. 
The  Bureau's  overall  goal  is  to  strengthen  and  advance  the  Nation's  science  and  technology 
and  facilitate  their  effective  application  for  public  benefit.  To  this  end,  the  Bureau  conducts 
research  and  provides:  (1)  a  basis  for  the  Nation's  physical  measurement  system,  (2)  scientific 
and  technological  services  for  industry  and  government,  (3)  a  technical  basis  for  equity  in 
trade,  and  (4)  technical  services  to  promote  public  safety.  The  Bureau's  technical  work  is  per- 
formed by  the  National  Measurement  Laboratory,  the  National  Engineering  Laboratory,  and 
the  Institute  for  Computer  Sciences  and  Technology. 

THE  NATIONAL  MEASUREMENT  LABORATORY  provides  the  national  system  of 
physical  and  chemical  and  materials  measurement;  coordinates  the  system  with  measurement 
systems  of  other  nations  and  furnishes  essential  services  leading  to  accurate  and  uniform 
physical  and  chemical  measurement  throughout  the  Nation's  scientific  community,  industry, 
and  commerce;  conducts  materials  research  leading  to  improved  methods  of  measurement, 
standards,  and  data  on  the  properties  of  materials  needed  by  industry,  commerce,  educational 
institutions,  and  Government;  provides  advisory  and  research  services  to  other  Government 
agencies;  develops,  produces,  and  distributes  Standard  Reference  Materials;  and  provides 
calibration  services.  The  Laboratory  consists  of  the  following  centers: 

Absolute  Physical  Quantities2  —  Radiation  Research  —  Thermodynamics  and 
Molecular  Science  —  Analytical  Chemistry  —  Materials  Science. 

THE  NATIONAL  ENGINEERING  LABORATORY  provides  technology  and  technical  ser- 
vices to  the  public  and  private  sectors  to  address  national  needs  and  to  solve  national 
problems;  conducts  research  in  engineering  and  applied  science  in  support  of  these  efforts; 
builds  and  maintains  competence  in  the  necessary  disciplines  required  to  carry  out  this 
research  and  technical  service;  develops  engineering  data  and  measurement  capabilities; 
provides  engineering  measurement  traceability  services;  develops  test  methods  and  proposes 
engineering  standards  and  code  changes;  develops  and  proposes  new  engineering  practices; 
and  develops  and  improves  mechanisms  to  transfer  results  of  its  research  to  the  ultimate  user. 
The  Laboratory  consists  of  the  following  centers: 

Applied  Mathematics  —  Electronics  and  Electrical  Engineering2  —  Mechanical 
Engineering  and  Process  Technology2  —  Building  Technology  —  Fire  Research  — 
Consumer  Product  Technology  —  Field  Methods. 

THE  INSTITUTE  FOR  COMPUTER  SCIENCES  AND  TECHNOLOGY  conducts 
research  and  provides  scientific  and  technical  services  to  aid  Federal  agencies  in  the  selection, 
acquisition,  application,  and  use  of  computer  technology  to  improve  effectiveness  and 
economy  in  Government  operations  in  accordance  with  Public  Law  89-306  (40  U.S.C.  759), 
relevant  Executive  Orders,  and  other  directives;  carries  out  this  mission  by  managing  the 
Federal  Information  Processing  Standards  Program,  developing  Federal  ADP  standards 
guidelines,  and  managing  Federal  participation  in  ADP  voluntary  standardization  activities; 
provides  scientific  and  technological  advisory  services  and  assistance  to  Federal  agencies;  and 
provides  the  technical  foundation  for  computer-related  policies  of  the  Federal  Government. 
The  Institute  consists  of  the  following  centers: 

Programming  Science  and  Technology  —  Computer  Systems  Engineering. 

'Headquarters  and  Laboratories  at  Gaithersburg,  MD,  unless  otherwise  noted; 
mailing  address  Washington,  DC  20234. 

'Some  divisions  within  the  center  are  located  at  Boulder,  CO  80303. 
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PREFACE 


TESTING  LABORATORIES  ARE  BEING  SUBJECTED  TO  EVER  INCREASING 
NUMBERS  OF  EXAMINATIONS,  AUDITS,  AND  INSPECTIONS  TO  ENSURE  THEIR 
TESTING  CAPABILITY,     TO  DATE,  OVER  60  PROGRAMS,  BOTH  PUBLIC  AND 
PRIVATE,  HAVE  BEEN  IDENTIFIED  WHICH  EVALUATE  TESTING  LABORATORIES 
AND  BESTOW  ACCREDITATION  IN  ONE  FASHION  OR  ANOTHER  FOR  TESTS  CON- 
DUCTED , 

THE  PAPERS  IN  THESE  PROCEEDINGS  WERE  PRESENTED  AT  A 
NATIONAL  CONFERENCE  ON  TESTING  LABORATORY  PERFORMANCE  EVALUATION 
AND  ACCREDITATION  HELD  AT  THE  NATIONAL  BUREAU  OF  STANDARDS  ON 
SEPTEMBER  25"26,  1979,     THE  PURPOSE  OF  THE  CONFERENCE  WAS  TO 
PROVIDE  A  FORUM  FOR  THE  DISSEMINATION  OF  INFORMATION  AND  THE 
DISCUSSION  OF  TECHNIQUES  AVAILABLE  FOR  EVALUATING  THE  PERFORMANCE 
OF  TESTING  LABORATORIES,  AND  ALSO  TO  PROVIDE  INFORMATION  ON  THE 
OPERATIONAL  ASPECTS  OF  SOME  EXISTING  AND  PROPOSED  ACCREDITATION 
PROGRAMS  AND  SYSTEMS, 

THE  CONFERENCE  WAS  ORGANIZED  THROUGH  THE  OFFICE  OF  TESTING 
LABORATORY  EVALUATION  TECHNOLOGY  AT  THE  NATIONAL  BUREAU  OF  STANDARDS 
BY  DR.  GERALD  A.  BERMAN  WHO  SERVED  AS  PROGRAM  COORDINATOR  AND 
GENERAL  CHAIRMAN,     THE  PROGRAM  CONSISTED  OF  29  PAPERS  WHICH  ADDRESSED 
VARIOUS  TECHNIQUES  FOR  EVALUATING  THE  PERFORMANCE  OF  TESTING  LABORA- 
TORIES, QUALITY  CONTROL  ASPECTS  OF  THE  TESTING  FUNCTION,  EXISTING 
AND  PROPOSED  ACCREDITATION  PROGRAMS  AND  SYSTEMS,  AND  INTERNATIONAL 
COORDINATION, 

EXCEPT  FOR  SOME  EDITORIAL  CHANGES,  THE  PAPERS  IN  THESE  PRO- 
CEEDINGS ARE  PRESENTED  AS  SUBMITTED  BY  THE  AUTHORS  AS  CAMERA-READY 
COPY, 
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OVERVIEW 
OF 

EVALUATION 

AND 

ACCREDITATION 


CHAIRMAN:    DR,  GERALD  A.  BERMAN 

NATIONAL  BUREAU  OF  STANDARDS 


National  Bureau  of  Standards  Special  Publication  591,  Proceedings  of  the  National  Conference 
on  Testing  Laboratory  Performance:  Evaluation  and  Accreditation,  held  at  NBS ,  Gaithersburg, 
MD,  September  25-26,  1979.        Issued  August  1980 


USER  EXPERIENCES  WITH  LABORATORY  ACCREDITATION 


James  E.  French 


Technical  Consultant 
Cortland,  NY 


Laboratory  accreditation  serves  a  two-fold  purpose  in  the 
marketplace.    This  relatively  new  technical  function  can  provide 
increased  confidence  in  the  capability  and  reliability  of  testing 
laboratories  by  customers  of  their  services.     The  function  can  also 
serve  as  a  marketing  tool  for  laboratory  managers.     In  this  sense 
there  is  an  analogy  with  product  certification.    Although  laboratory 
accreditation  may  eventually  become  a  leveling  force,  much  as 
accreditation  of  hospitals  and  academic  departments  within  colleges 
has  become,  the  enterprising  laboratory  manager  should  not  regard 
accreditation  as  a  least  common  denominator  activity  at  this  time. 
Experience  of  testing  laboratories  in  several  disciplines  where 
accreditation  has  been  available  or  required  are  reviewed.  Long 
range  roles  for  the  accrediting  function  are  discussed. 

Key  words:     Accreditation;  certification;  colleges;  confidence; 
criteria;  evaluation;  hospitals;  inspectors;  marketing;  qualification; 
re-examination;  technicians. 


1.  Introduction 

Accreditation  of  testing  laboratories 
by  Third-party  organizations  is  a  relatively 
new  function  in  the  marketplace  for  technical 
services.     For  purposes  of  this  presentation, 
Laboratory  accreditation  shall  be  charac- 
terized as  including  technical  and  adminis- 
trative evaluations  with  periodic  re- 
examinations.    Although  some  accreditation 
activities  have  considerable  historical 
continuity,  most  of  those  activities  stimu- 
lating interest  in  laboratory  managers  are 
of  recent  origin.    The  characteristics  of 
laboratory  accreditation  activities  mentioned 
here  are  not  exhaustive,  but  need  not  be  in 
order  to  review  the  perception  of  this  new 
function  among  its  users. 

2.    User  perceptions 
of  laboratory  accreditation 

The  direct  users  of  laboratory  accredi- 
tation are  the  testing  laboratories  them- 
selves.    The  beneficiaries  include  several 


levels  of  users  of  laboratory  services. 
These  will  be  discussed  under  trends. 

Informal  surveys  and  discussions  with 
laboratory  managers  reveal  that  those  who 
have  no  prior  experience  are  wary  of  the  cost/ 
benefit  factor.     Industry  executives  presently 
seem  suspicious  that  accreditation  will  have 
a  least  common  denominator  or  leveling  effect. 
This  reaction  has  been  noted  with  regard  to 
other  forms  of  standardization  especially  so 
among  scientist/businessmen.  Laboratory 
managers  who  are  looking  to  the  future  see 
accreditation  as  a  long  range  marketing  tool. 
They  know  that  it  will  be  difficult  to  imple- 
ment on  a  broad  basis,  difficult  to  achieve, 
and  difficult  to  market  of  itself.     In  a 
complementary  sense,  these  same  industry 
leaders  realize  that  the  ability  to  compare 
competing  service  organizations,  the  presence 
of  external  monitors  for  laboratory  quality 
control,  and  published  statements  about  the 
capabilities  of  testing  laboratories  based 
on  common  factors  will  provide  a  new  disci- 
pline as  the  economy  relies  increasingly  on 
scientific  testing. 


3.     Analogy  with  product  certification 

The  modern  marketplace  demands  "both 
products  and  services .     Each  category  is 
becoming  more  complex  as  it  responds  to 
marketplace  needs.     Consumers,  he  they 
residential,  industrial,  commercial,  or 
institutional,  increasingly  need  evidence  to 
support  their  confidence  in  the  products  and 
services  which  they  purchase. 

The  response  to  this  need  for  product 
confidence  has  heen  product  certification. 
This  activity  is  conducted  hy  independent 
laboratories ,  trade  associations,  model  code 
bodies  and  certification  councils  operating 
under  either  commercial  or  governmental  man- 
date.    Many  of  these  programs  provide  market- 
ing tools  within  certain  constraints.  As 
such  activities  become  established  and  well- 
known,  there  seems  to  be  an  increasing 
reliance  on  the  information  which  they 
provide  to  consumers . 

The  response  to  the  need  for  supporting 
evidence  with  respect  to  testing  laboratory 
services  is  laboratory  accreditation.  When 
this  activity  is  based  on  consensus  criteria, 
evaluation  by  qualified  inspectors,  and 
periodic  re-examination  the  analogy  with 
product  certification  holds.     The  voids  in 
completing  the  comparison  of  the  two  systems 
are  accreditation  of  certifiers,  on  one  hand, 
and  qualification  of  inspectors  on  the  other. 
With  respect  to  product  certification  there 
tends  to  be  some  confusion  between  accredi- 
ting laboratories  and  accrediting  certifiers. 
Evaluation  of  an  independent  laboratories 
technical  capabilities  may  not  include 
evaluation  of  the  administrative  capabilities 
necessary  for  certification.     The  general  lack 
of  this  latter  service  particularly  with 
respect  to  product  safety  certifiers  has 
impeded  the  market  acceptance  of  alternate 
sources  of  certification.     Since  some  labora- 
tory accreditation  activities  have  been 
informal  to  date,  only  casual  attention  has 
been  paid  to  formalizing  the  qualification 
of  inspectors  for  particular  kinds  of 
laboratories.     On  the  record  the  budding 
national  laboratory  accreditation  programs 
appear  to  be  making  effort  to  document  these 
qualifications  and  harmonize  them. 

k .     Analogy  with 
college  and  hospital  accreditation 

Institutional  accreditation  is  not  a 
new  activity  in  the  marketplace  for  services. 
Although  the  need  for  accreditation  of 
testing  laboratories  is  being  established 
in  present  time,  other  service  organizations 
have  been  subjected  to  such  evaluation  for 
many  years .     The  efficacy  of  accreditation 


for  colleges  and  hospitals  is  taken  for 
granted  to  such  an  extent  that  many  consumers 
of  their  services  are  almost  unaware  of  the 
confidence  which  they  obtain  through  accredi- 
tation.    Other  forces  in  the  marketplace  for 
health  care  and  educational  services  are, 
however ,  well  aware  of  the  importance  of 
accreditation.     The  many  departments  in  a 
hospital,  including  its  laboratories,  must 
be  evaluated  according  to  criteria  well- 
established  in  the  medical  community  and  be 
inspected  by  peers,  on  a  regular  basis.  The 
insurance  industry  seeks  this  supporting 
evidence  for  its  confidence  in  hospitals.  On 
a  periodic  basis  the  academic  departments  of 
a  college  or  university  are  evaluated  by  their 
peers.     Confidence  in  their  diplomas  by  major 
employers  of  their  graduates  is  sustained 
through  accreditation.     The  fact  that  there 
may  be  some  elements  of  professional  disci- 
pline in  these  fields  is  known.  Laboratory 
accreditors  should  be  aware  of  this  risk  also. 

The  fact  that  accreditation  of  hospitals 
and  colleges  is  only  barely  perceived  by  the 
consuming  public,  except  when  a  withdrawal 
makes  local  news,  and  that  nearly  all  insti- 
tutions maintain  accreditation  once  it  is 
obtained  makes  some  managers  of  commercial 
testing  laboratories  believe  that  the 
accreditation  activity  has  a  levelling  effect 
thus  detracting  from  their  innovation.  This 
belief  should  not  become  a  value  assumption 
because  the  effect  is  still  manageable  at 
this  point  in  time  while  policies  and  criteria 
are  still  being  formulated. 

5.  Typical  markets  where  laboratory 
accreditation  is  necessary  or  useful 

Laboratory  accreditation  has  been  estab- 
lished in  certain  disciplines  for  a  number 
of  years.     Several  of  these  programs  will  be 
discussed  in  detail  during  the  conference, 
but  I  wish  to  establish  the  rather  high 
degree  of  user  satisfaction  with  these  pro- 
grams in  the  markets  which  they  now  serve. 
The  shortcomings  of  any  particular  program  as 
viewed  by  laboratory  managers  will  not  be 
reviewed  in  the  presentation. 

Accreditation  of  a  construction 
materials  testing  (CMT)  laboratory  by  the 
Cement  and  Concrete  Reference  Laboratory 
(CCRL)  Program  is  now  often  specified  in 
construction  contracts.     This  recognition 
has  come  about  in  part  through  the  recommen- 
dation by  those  laboratories  which  have 
passed  this  muster.     CCRL  accreditation 
represents  a  kind  of  industry-wide  quality 
assurance  which  establishes  confidence  among 
both  laboratories  and  their  customers. 


The  College  of  American  Patholigists 
(CAP)  Accreditation  Program  is  ■basically- 
oriented  toward  clinical  service  laboratories. 
Some  time  after  its  establishment,  the 

j  program  was  expanded  to  admit  the  development 
laboratories  of  medical  device  manufacturers . 
In  the  experience  of  one  particular  firm, 

I  this  accreditation  enabled  it  to  participate 
in  collaborative  reference  studies  to  con- 

I  firm  performance  of  a  new  product.  The  other 
CAP-accredited  laboratories  would  have  been 

j  less  interested  in  participating  in  that 
exercise  had  the  manufacturer  not  also  been 

J  CAP-accredited . 

In  the  discipline  of  non-destructive 
testing  (NDT),  full-scale  laboratory  accredi- 

i  tation  is  not  yet  feasible;  however,  certifi- 
cation of  NDT  technicians  has  been  the  basis 
for  reliability  on  the  technology  for  several 

j  years.     Laboratories  offering  NDT  services 
are  expected  to  maintain  staff  certification, 

!    and  testing  contracts  so  specify.  This 
particular  approach  has  provided  mobility 

j    among  technicians  at  the  working  level  during 

j  a  period  of  rapid  growth  in  the  discipline. 
Some  form  of  technician  certification  ought 
to  be  coupled  with  laboratory  accreditation 
so  that  personnel  changes  can  occur  with 
minimum  effect  on  the  organization's 
accreditation. 

While  most  accreditation  programs  are 
national  in  coverage,  some  activities  are 
'  conducted  on  a  regional  or  local  basis 
generally  because  of  marketing  needs . 
Examples  include  state  and  municipal  level 
recognition  of  laboratories  to  inspect  and 
test  specialized  electrical  installations  or 
pressure  vessels  not  previously  code- 
certified.     These  activities  respond  to  the 
needs  of  laboratories  as  well  as  the  regulated 
parties  although  not  as  formal  as  they  should 
be,  regional  accreditation  programs  through 
their  existence  and  user  satisfaction  demons- 
trate the  high  potential  value  of  this  vital 
new  service  in  the  contemporary  economy. 

6.     Future  trends 

The  most  significant  trend  yet  to  come 
in  the  field  of  laboratory  accreditations 
is  the  mechanism  for  identifying  and  respond- 
ing to  the  critical  mass  of  interest  for  the 
development  of  criteria  for  accreditation 
of  a  given  class  of  laboratories.     One  hopes 
that  with  maturation  of  this  activity,  the 
process  of  accreditation  will  eventually 
become  straight  forward  dependent  on  the 
specific  criteria  applicable  to  particular 
laboratories.     This  condition  applies  whether 
the  basis  of  accreditation  is  by  product  or 
by  discipline  or  a  combination. 


The  process  of  criteria  development  must 
anticipate  customer  expectations  for  the 
conduct  of  laboratory  services,  much  the 
same  as  this  sort  of  reasoning  is  being 
applied  to  the  development  of  product 
standards .     Involvement  of  regulatory 
authorities  who  are  likely  to  recognize 
accreditation  in  their  jurisdictions  will 
also  become  increasingly  important  as  more 
aspects  of  product  performance  and  safety 
become  subject  to  demonstrated  scientific 
testing. 

Accrediting  organizations  must  not  be 
aloof  from  the  impact  of  their  activities, 
I  believe  they  have  a  responsibility  to 
assist  in  promoting  their  activities  through 
public  awareness  of  the  criteria  and  wide 
dissemination  of  their  directories.  They 
must  also  grow  out  of  the  fear  of  advertising 
by  accreditees.     It  is  unnecessary,  in  my 
opinion,  to  believe  that  a  person  contracting 
for  laboratory  services  is  going  to  be  duped 
into  thinking  that  an  accreditation  will 
cover  up  otherwise  incompetent  conduct.  In 
this  regard  laboratory  accreditors  can  learn 
a  great  deal  from  the  experience  of  college 
and  hospital  accreditors.     From  the  user's 
standpoint,  international  cooperation  is  a 
vital  counterpart  to  a  domestic  program. 
Without  it,  much  of  the  progress  in  inter- 
national trade  accomplished  through  the  multi- 
lateral trade  negotiations  will  be  set  back. 

In  conclusion,  laboratory  accreditation 
can  serve  as  a  stimulus  to  the  market  for 
testing  services  for  a  long  time  before  it 
dampens  the  field  through  over-regulation. 
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Laboratory  accreditation  systems  which  formally  determine  and  recognize 
that  a  laboratory  has  the  capability  to  carry  out  specific  tests  or  types  of 
tests  are  increasing  both  in  number  and  in  the  number  of  laboratories  examined 
and  accredited.  The  need  for  such  systems  can  be  traced  to  the  growing  need 
for  laboratory  testing  in  general.  These  systems  are  being  developed  normally 
to  facilitate  both  national  and  international  trade.  Fifty-six  laboratory 
accreditation  systems  were  recently  examined  in  a  Department  of  Commerce 
study.  Only  2  of  the  systems  existed  in  1947.  By  1970  the  number  had  grown 
to  33,  and  by  1978  the  number  was  56  with  a  significant  portion  of  this 
increase  occuring  in  1977  and  1978.  Over  5,500  laboratories  are  formally 
recognized  by  these  systems  and,  since  many  of  the  systems  are  new,  this  num- 
ber should  increase  substantially  in  the  80's.  There  is  also  a  growing  inter- 
est in  the  international  recognition  of  national  accreditation  systems. 
Public  and  private  sector  coordination  to  promote  acceptance  of  accreditation 
criteria  and  consolidation  of  accreditation  systems  is  a  growing  need. 
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1.  Introduction 

In  a  way,  this  whole  conference  is  on 
the  state-of-the-art  in  laboratory 
accreditation.  General  descriptions  of 
laboratory  accreditation  models  will  be 
presented.  Descriptions  of  evaluation 
technology  which  form  part  of  laboratory 
accreditation  models  will  be  presented. 
Specific  accreditation  systems  will  be 
described.  Future  concepts  will  be  pre- 
sented. And  finally,  the  program  will 
conclude  with  a  briefing  on  an  upcoming 
conference  focusing  on  international  co- 
ordination of  other  accreditation  systems. 

The  state-of-the-art  I  intend  to 
present  will  deal  with  the  laboratory 
accreditation  systems  in  existence  in  the 
United  States  at  this  time. 


2.  Definitions 

Before  describing  the  extent  of  labor- 
atory accreditation  in  the  United  States, 
I  would  like  to  start  with  a  definition. 
Laboratory  accreditation  is  the  formal 
determination  and  recognition  that  a 
testing  laboratory  has  the  capability  to 
carry  out  specific  tests  or  types  of 
tests.  Under  this  definition  laboratory 
accreditation  does  not  include  the 
development  or  promulgation  of  test 
methods  or  standards.  Laboratory  accredi- 
tation under  this  definition  also  does  not 
include  the  act  of  certifying  that  a 
product  meets  a  standard  since  such  a  cer- 
tification requires  attention  to  many  more 
aspects  in  the  production  process  than 
simply  to  the  testing  function. 
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3.  Need  for  Laboratory  Accreditation 

Laboratory  accreditation  systems  have 
been  developed  to  improve  the  ability  of 
testing  organizations  to  meet  a  wide 
variety  of  needs.  First,  there  is  the 
need  of  government  to  assure  compliance  to 
health  and  safety  standards.  Then  there 
is  the  need  of  procurement  authorities  to 
determine  that  a  product  received  meets 
the  requirements  specified  in  the  purchase 
request.  Producers  and  manufacturers 
often  must  establish  the  specific  char- 
acteristics and  performance  of  their 
products  as  measured  by  some  test  method. 
Health  service  agencies  need  accurate 
testing  information  to  diagnose  potential 
problems  and  take  corrective  actions. 
Consumers  are  more  and  more  interested  in 
specific  performance  characteristics  and 
ways  to  measure  differences  among  pro- 
ducts. General  contracting  groups  such  as 
in  the  building  trades  must  meet  certain 
specifications  which  require  definitive 
test  information.  Finally,  many  certify- 
ing agencies  have  been  established  to 
attest  to  the  accurate  performance  of 
products  and  testing  is  an  integral  part 
of  that  determination. 

The  main  objective  of  laboratory 
accreditation  systems  is  to  facilitate 
trade  both  nationally  and  internation- 
ally. Over  and  above  this  objective,  how- 
ever, laboratory  accreditation  is  used  to 
meet  other  objectives,  including  ensuring 
validity  of  test  data,  promoting  wide 
acceptance  of  test  data,  providing  more 
efficient  use  of  testing  facilities, 
giving  credibility  to  more  use  of  testing 
laboratories,  giving  additional  status  to 
competent  laboratories,  promoting  good 
testing  practices,  improving  test  methods, 
and  providing  information  about  accredited 
laboratories.  Such  programs  inherently 
will  upgrade  the  quality  of  measurement  in 
testing  laboratories.  They  will  emphasize 
the  need  for  calibration  services,  provide 
models  for  good  laboratory  management  per- 
formance, and  aid  in  the  improvement  of 
quality  control  in  manufacturing.  In 
addition,  laboratory  accreditation  will  be 
an  aid  to  the  reduction  of  defective 
products  and  a  reduction  in  personal 
injury,  property  damage,  and  product 
liability  suits. 

4.  Approaches  to  Laboratory 
Accreditation  Systems 

There  are  two  basic  approaches  to 
accreditation  of    laboratories.     First  is 
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the  approach  where  laboratories  are 
accredited  to  test  specific  products  in 
conformance  to  standards  using  test 
methods  pertinent  to  that  product.  Such 
product-focused  systems  are  designed  for 
the  users  of  laboratories,  particularly 
for  those  desiring  a  certification  that 
products  meet  certain  standards.  These 
systems  typically  identify  the  testing 
methods  to  be  included  in  the  program 
before  accepting  applications  for  accredi- 
tation. The  programs  provide  accredita- 
tion for  only  a  limited  number  of  test 
methods  although  they  effectively  serve 
the  needs  of  laboratories  for  recognition 
to  a  particular  set  of  clients. 

Second  is  the  approach  where  labora- 
tories are  accredited  to  conduct  tests  in 
broad  technical  areas  or  groups.  This 
discipline-focused  approach  accepts  a 
laboratory's  statement  of  tests  it  can 
perform  and  then  develops  procedures  to 
verify  the  laboratory's  capability  to  per- 
form the  tests.  The  general  area  of 
accreditation  is  announced,  but  the  speci- 
fic tests  to  be  verified  will  depend  upon 
the  laboratory's  testing  programs  and  the 
accreditor's  ability  to  examine  the  skills 
of  the  laboratory  to  perform  those  testing 
programs.  Such  programs  are  advantageous 
to  the  laboratories  since  accreditation 
can  be  granted  for  a  large  spectrum  of 
test  methods  but  are  often  difficult  for 
users  of  laboratories  to  use  since  the 
accredited  tests  may  often  not  be  in  the 
language  of  the  product  performance  char- 
acteristics. 

5.    Operation  of  Laboratory 
Accreditation  Systems 

Most  laboratory  accreditation  systems 
contain  the  following  features: 

A.  They  obtain  objective  information 
about  the  testing  laboratory's 
capabilities  either  through  a 
questionnaire  or  an  application 
form. 

B.  Each  laboratory  is  visited  and  its 
operations  reviewed  by  an  evalua- 
tion team. 

C.  Proficiency  tests  are  established 
for  certain  tests  and  the  labora- 
tory required  to  participate  so 
that  performance  of  the  various 
laboratories  can  be  compared. 


D.  After  evaluation  of  the  results  of 
the  first  three  features,  a  formal 
recognition  of  the  laboratory's 
capability  is  granted. 

E.  The  laboratory's  performance  is 
reassessed  periodically  to  assure 
continued  compliance  with  the 
criteri  a. 

F.  The  accreditation  system  makes 
known  to  potential  users  the  cap- 
abilities of  potential  labora- 
tories in  its  system. 

6.    Laboratory  Accreditation  Programs 
in  the  U.S. 

Mr.  Charles  Hyer,  in  a  study  performed 
for  the  Department  of  Commerce  in  1978, 
examined  56  public  laboratory  accredita- 
tion programs  in  existence  in  the  U.S. 
These  programs  were  equally  divided  among 
Federal  agencies,  state  and  local 
agencies,  and  trade  associations.  The 
rapid  increase  in  the  establishment  of 
these  programs  is  illustrated  in 
Exhibit  1.  Apparently  laboratory  accredi- 
tation programs  fill  a  need  because  the 
programs  have  been  established  recently  at 
an  increasing  rate.  The  number  of  labora- 
tories formally  recognized  by  these 
systems  is  growing  at  an  even  faster  rate 
as  shown  in  Exhibit  2.  In  this  Exhibit 
the  number  of  laboratories  granted 
accreditation  in  1978  was  plotted  versus 
the  year  the  accreditation  program  began. 
Typically,  these  programs  require  several 
years  before  significant  numbers  of  labor- 
atories become  accredited.  With  the  large 
number  of  programs  initiated  in  the 
mid-70' s,  it  would  appear  that  the  number 
of  laboratories  accredited  will  be  in- 
creasing very  rapidly  in  the  80' s. 

With  the  increasing  number  of  accredi- 
tation programs,  we  are  beginning  to 
observe  a  situation  where  laboratories  are 
accredited  by  more  than  one  system.  This 
overlap  and  duplication  in  the  examination 
of  laboratories  is  just  beginning  to  be 
felt.  For  example,  one  laboratory  pro- 
vides operating  space  for  numerous  differ- 
ent accreditation  agents  at  its  facility, 
and  this  operating  space  is  almost  con- 
tinually utilized.  In  other  cases,  we 
know  of  laboratories  being  accredited  by 
30  or  40  systems.  It  would  appear  that 
this  overlap  and  duplication  is  costly  and 
burdensome  and  will  grow  substantially  in 
the  coming  years,  and  it  is  one  reason  why 
some  sort  of    a    national    coordination  of 


laboratory  accreditation  systems  appears 
to  be  desirable. 

7.  Toward  "Nationally  Recognized" 
Laboratories 

Another  factor  leading  to  the  promul- 
gation of  laboratory  accreditation  systems 
is  the  enactment  of  laws  and  statutes 
which  specify  that  testing  must  be  done  by 
"nationally  recognized"  laboratories. 
Private  groups  may  very  well  develop 
systems  which  can  be  recognized  nationally 
by  Federal  agencies,  or  Federal  agencies 
and  private  groups  may  develop  systems 
which  warrant  national  recognition.  The 
demand  for  national  recognition,  however, 
will  tend  to  increase  the  pressure  for  the 
development  of  broad-based  laboratory 
accreditation  systems  capable  of  multiple 
product  testing  and  evaluation,  or  at  a 
minimum,  effective  coordination  of  such 
systems  in  the  interest  of  economy. 

Each  laboratory  accreditation  system 
has  its  own  specific  criteria  which  it 
will  use  to  evaluate  the  testing  labora- 
tories. We  have  examined  many  of  these 
criteria;  and  although  it  can  be  said  that 
in  general  the  criteria  specify  similar 
requirements,  the  laboratories  find  that 
fulfilling  the  requirements  of  these 
criteria  calls  for  individual  effort  for 
each  specific  criterion.  With  the  in- 
crease in  laboratory  accreditation  systems 
there  will  be  a  similar  increase  in  the 
work  that  laboratories  must  perform  in 
order  to  be  accredited  by  these  various 
systems.  Accordingly,  some  attempt  needs 
to  be  made  to  arrive  at  universally  accep- 
table criteria  for  application  in  poten- 
tially duplicative,  and  perhaps  conflict- 
ing, programs.  We  believe  that  this  is  an 
appropriate  subject  for  the  next  national 
conference  on  laboratory  accreditation. 

8.  International  Implications 

International  laboratory  accreditation 
conferences  have  been  held  in  1977  and 
1978.  A  third  one  is  scheduled  for 
October  of  this  year  and  a  fourth  one  is 
planned  for  the  fall  of  1980.  The  purpose 
of  these  International  Laboratory  Accredi- 
tation Conferences  (ILAC)  is  to  examine 
ways  in  which  laboratory  accreditation 
systems  in  the  various  participating  coun- 
tries can  be  multilaterally  recognized. 
If  such  recognition  can  be  established, 
then  products  tested  in  one  country  may  be 
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accepted  in  another  country  without  re- 
testing.  The  formal  international  recog- 
nition of  accreditation  programs  will,  of 
course,  depend  on  treaties  or  other 
arrangements  made  among  the  participating 
countries.  In  the  interim,  ILAC  is 
attempting  to  prepare  a  Directory  of 
Laboratory  Accreditation  Systems  through- 
out the  world.  In  an  attempt  to  assure 
some  sort  of  minimum  acceptable  character- 
istics of  these  accreditation  systems, 
ILAC  will  be  specifying  minimum  criteria 
which  are  to  be  met  by  a  laboratory 
accreditation  system  in  order  for  that 
system  to  be  listed  in  the  Directory. 
Some  programs  already  in  existence  in  the 
United  States  may  not  be  accepted  for 
listing  in  this  Directory  because  they  do 
not  meet  the  criteria  being  suggested. 
More  on  this  matter  will  be  presented  in 
the  last  session  of  this  conference. 
Suffice  it  to  say  that  international 
coordination  of  laboratory  accreditation 
is  of  growing  interest. 

9.  Summary 

U.S.  laboratory  accreditation  systems 
are  growing  rapidly  in  number  as  are  the 
number  of  laboratories  actually  recog- 
nized. There  are  a  number  of  forces  in 
existence  both  nationally  and  interna- 
tionally which  are  beginning  to  focus  on 
the  need  to  coordinate  these  systems.  The 
key  to  such  coordination  will  be  the 
development  of  a  universally  acceptable 
!  criteria  for  evaluating  the  testing 
laboratories,  a  public  and  private  sector 
coordinating  body  to  promote  acceptance  of 
the  criteria,  and  the  consolidation  of 
accreditation  systems. 
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EXHIBIT  1 

NUMBER  OF  ACCREDITATION  PROGRAMS  IN  THE  U.S. 

Ref.:    C.  W.  Hyer,  "Principal  Aspects  of 
U.S.  Laboratory  Accreditation  Programs,"  January  24,  1979 

60 


1945  50  55  60  65  70  75  80 

Year  Accreditation  Program  Began 
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The  paper  provides  an  overview  of  generic  criteria  for  laboratory 
capability  evaluation.     The  new  American  National  Standard,  ANSI/ASTM 
E  5^8-T95  Standard  Practice  for  Use  in  the  Evaluation  of  Testing  and 
Inspection  Agencies,  is  reviewed.     It  provides  generic  criteria  for 
evaluation  of  an  agency  for  intended  purpose,  agency  organization, 
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standards  for  specific  disciplines  and  product  areas.     Discussion  is 
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1.  Introduction 

Generic  criteria  are  required  to  permit 
development  of  a  uniform  base  in  preparing 
specific  working  requirements  to  evaluate 
testing  and  inspection  agencies.  These 
generic  criteria  are  provided  in  ANSI/ASTM 
E  51+8-T9,  Standard  Practice  for  Use  in  the 
Evaluation  of  Testing  and  Inspection  Agencies 
(Ref  l).     This  standard  serves  as  a  starting 
point  to  achieve  effective  and  efficient 
review  of  test  and  inspection  organizations 
and  relates  to  criteria  for  the  evaluators 
and  the  evaluation  system.     In  addition  the 
generic  criteria  are  useful  in  developing  a 
base  for  laboratory  accreditation  activity. 

2.  Needs 

Business,  legal,  and  standards  practice 
usually  require  final  product  or  service  to 
be  evaluated  for  performance  against  estab- 
lished requirements  and  compliance  criteria. 
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However,  in  many  areas  it  may  be  desirable 
for  additional  information  to  be  known  about 
an  actual  test  or  inspection  process,  through 
evaluation  of  agencies  or  activities  in- 
volved in  earlier  decision  making  processes. 

In  normal  commercial  practice  this  flow 
of  information  and  evaluation  is  considered 
in  the  regular  contract  base,  the  need  to 
know,  and  with  full  regard  for  associated 
proprietary  information.     However,  with  ex- 
panding national  and  international  trade, 
and  involvement  of  third  parties  in  commerce, 
it  is  desirable  for  ground  rules  to  be  de- 
fined to  aid  in  evaluations  and  possible  ac- 
creditation, especially  with  indirect  and 
complex  procurement  arrangements.     Also,  it 
may  well  be  important  for  regulatory  re- 
quirements to  relate  their  specific  criteria 
to  such  defined  normal  commercial  practice. 
Such  action  may  result  in  the  least  adverse 
effect  on  commerce  and  productivity  with 
reduction  in  possible  regulatory  liability 
(Ref  2). 


Thus  the  need  for  the  formation  of  ASTM 
Technical  Committee  E-36,  on  Criteria  for 
the  Evaluation  of  Testing  and  Inspection 
Agencies,  and  with  the  development  of  ASTM 
E  5^+8-79  Standard  Practice  Generic  Criteria. 

3.  Considerations 

With  the  disengagement  in  the  1960's  of 
the  Department  of  Defense  from  on-site  test 
and  inspection,  there  was  a  formal  trend  to 
improving  visibility  of  quality  and  relia- 
bility information  for  products  and  services 
from  suppliers.     An  example  of  action  taken 
to  improve  this  flow  of  data  and  accountabil- 
ity was  the  Electronic  Industries  Association 
development  of  Standard  EIA-1000,  EIA  Stan- 
dard Procedure  and  Questionnaires  for  Elec- 
tronic and  Electrical  Parts  Suppliers 
Quality  and  Reliability  Assurance  (Ref  3). 
This  provided  a  base  for  buyers  to  obtain 
quick  answers  on  questions  before  purchasing 
electronic  parts.     The  system  was  voluntary 
and  also  included  provision  for  sharing  of 
system  reports. 

Recognizing  a  need  for  furthering  this 
evaluation  in  many  test  and  inspection  areas, 
ASTM  Technical  Committee  E-36  was  formed  to 
prepare  standard  consensus  generic  criteria. 
These  criteria  were  to  be  used  by  others  in 
preparing  specific  requirements  for  qualifi- 
cation and  accreditation  of  testing  and  in- 
spection agencies.     The  Committee  Scope  in- 
cluded preparing  generic  standards  on  its 
own  and  preparing  specific  standards  when 
not  in  area  of  interest  of  other  technical 
committees  and  when  coordination  is  needed 
with  them.     Also  the  Committee  Scope  included 
standard  nomenclature  and  definitions,  aid- 
ing and  advising  ASTM  technical  committees 
in  the  preparation  of  standards,  correlation 
and  consolidation  of  similar  standards  pre- 
pared by  separate  committees,  and  institut- 
ing cooperation  between  these  technical 
committees  in  areas  of  common  interest. 

Further,  the  Committee  was  to  encourage, 
secure,  and  promote  cooperation  between  ASTM 
and  other  organizations  including  those  that 
may  accredit  testing  and  inspection  agencies. 
And,  the  Committee  was  to  advance,  improve, 
standardize,  and  unify  such  standards,  in- 
cluding liaison  with  government  and  other 
organizations . 

This  presentation  is  not  intended  as  a 
formal  report  on  the  work  of  ASTM  E-36. 
Rather  it  is  intended  for  those  not  familiar 
with  the  voluntary  consensus  system  to  aid 
in  the  understanding  of  the  responsive 
nature  of  this  work  and  its  subsequent  value 
through  application  of  specific  criteria  for 

12 


products  and  services  in  the  voluntary  syste 
and  also  as  appropriate  in  the  mandatory  reg 
ulatory  system.     Therefore  this  is  not  a  ful 
report  and  those  parties  interested  in  more 
factual  details  should  refer  to  the  writer 
and  the  Committee. 

k.     Test  and  Inspection  Agencies 

The  generic  criteria  in  Standard  E  5^+8- 
79  replaces  the  earlier  edition  of  Standard 
E  5^+8-77.     For  actual  use  this  standard 
should  be  interpreted  by  a  technical  stan- 
dard for  each  specific  product,  service,  or 
test  method.     This  is  obvious  since  no  sin- 
gle standard  could  be  directly  applied  with 
full  value  in  all  possible  applications  and 
conditions.     For  example  certain  industries 
have  special  practices  and  requirements 
which  must  be  placed  into  perspective.  Even 
with  such  additional  work  there  is  much  to 
be  gained  through  the  consideration  of  out- 
line format,  applicable  terms,  definitions, 
and  possibly  wording  generated  in  the  gener- 
ic standard.     Final  decision  for  application 
must  rest  with  the  technical  committee  re- 
sponsible for  the  final  document. 

Likewise  care  must  be  used  if  the  stan- j 
dard  is  applied  in  normal  direct  negotiations 
and  in  dealings  between  buyers  and  sellers 
where  there  may  be  special  economic  and  pro- 
prietary concerns. 

5.     E  5^8-79  Criteria 

Although  only  the  actual  Standard  may 
be  used  for  full  and  proper  understanding, 
this  paper  calls  attention  to  certain  areas 
which  are  covered  in  detail  in  E  5^+8-79  • 

First,  the  purpose  for  evaluation 
should  be  identified  and  considered  in  de- 
veloping any  specific  document  from  this 
generic  base.     Obviously  this  purpose  must 
identify  the  system  in  which  the  specific 
document  will  be  used  and  criteria  relating 
to  evaluator  or  accreditor.     Without  this 
understanding  it  would  be  impossible  to 
properly  apply  specific  requirements  in  the 
test  and  inspection  environment. 

Likewise  the  specific  criteria  must  be 
clear  in  relating  to  possible  implications 
in  past,  present,  and  future  tense  of  the 
application. 

Second,  the  agency  organization  should 
be  considered  for  necessary  identification, 
affiliation,  history,  services,  and  past 
recognition  achievements.     Here  the  intent 
is  to  provide  only  that  information  which 
must  be  considered  necessary  in  specific 
applications . 


Third,  the  human  resources  of  an  agency 
should  he  presented  to  relate  personnel,  re- 
sponsibility, and  authority  with  appropriate 
job  descriptions  and  identification  of  means 
for  maintaining  personnel  qualifications, 
work  experience,  training  history,  and  facts 
on  ensuring  continued  competence  of  person- 
nel. 

Fourth,  the  material  resources  of  an 
agency  should  he  identified  relating  to 
I  actual  equipment  and  facilities,  calibration 
standards,  and  other  information  necessary 
to  ensure  update  and  service. 

Fifth,  the  quality  system  of  an  agency 
should  identify  the  procedural  systems  af- 
fecting the  quality  of  services  with  details 
on  rationale  of  the  metrology  system  and 
support  activities  of  calibration,  traceabil- 

j ity,  data  analysis,  feedback,  and  necessary 
survey  and  audit  activities  with  periodic 

i on-site  reviews. 

In  summary,  the  standard  provides  a 
I  comprehensive  format  and  approach,  useful 
internally  as  well  as  by  prospective  out- 
I  siders,  where  there  is  proper  specific  stan- 
'  dard  consideration. 

6.  Evaluation  System 

Obviously  the  proper  application  of 
>  E  5^-8-79  requires  a  full  understanding  for 
1  the  associated  evaluation  system  and  if 
applicable  the  full  details  on  any  addition- 
al accreditation  system. 

Although  not  covered  in  detail  in  the 
I  standard  it  is  assumed  there  would  be  pro- 
1  per  consideration  given  in  the  system  to 
both  prevent  the  generation  of  incorrect 
I  reports  and  to  provide  effective  and  effi- 
cient action  should  there  be  any  questions 
or  need  for  control. 

7.  Evaluator  Criteria 

Likewise,  the  standard  does  not  identi- 
fy requirements  for  the  evaluator  and  any 
associated  accreditors.  Here  again  it  is 
expected  such  activities  will  be  properly 
qualified  and  under  supervision  to  assure 
effective  and  efficient  operation,  both  for 
the  evaluator  and  the  agency  being  evalu- 
ated. 

8.  Specific  Criteria 

The  continuing  effort  of  Technical 
Committee  E-36  has  indeed  been  rewarding. 
Through  this  voluntary  consensus  base  there 
have  been  developed  various  specific  criteria 


standards.     For  example,  in  the  Manufactured 
Housing  area  there  are  ASTM  Standards  E  5^2 
and  E  651.     In  the  government  voluntary  area 
there  is  the  Department  of  Commerce  National 
Voluntary  Laboratory  Accreditation  Program 
(NVLAP).     And  in  addition  there  are  several 
possible  mandatory  government  applications 
in  development,  all  based  on  E  5^+8-79.  How- 
ever, the  subject  of  this  paper  is  generic 
criteria  and  further  details  must  be  the 
subject  of  other  presentations. 

9-     International  Applications 

The  work  of  E-36  has  received  aid  from 
abroad,  both  in  the  preparation  of  the  E 
5^8-79  and  in  its  application  in  other  na- 
tional criteria  standards  and  evaluation 
systems.     It  is  expected  this  cooperation 
will  continue.     There  is  also  some  similar 
effort  in  international  standards  bodies  and 
in  ad  hoc  activities.     Again,  these  are  be- 
yond the  scope  of  this  paper  except  to  note 
that  this  cooperation  is  expected  to  aid  de- 
velopment of  further  generic  and  specific 
criteria  in  areas  of  products,  services,  and 
evaluation  or  accreditation  systems. 

10.     Evaluation  Effectiveness  and 
Efficiency 

The  development  of  E  5^8-79  has  been 
accomplished  with  full  appreciation  of  de- 
mands for  possible  reviews,  paperwork,  and 
delays  in  the  effectiveness  and  efficiency 
of  normal  productive  operations.  Recogni- 
tion of  the  need  for  criteria  to  be  cost 
effective  requires  full  consideration  by 
users  of  the  standard  and  in  development  of 
any  associated  specific  criteria. 

Special  attention  has  been  given  to 
setting  up  a  structure  that  will  be  flexible 
with  time  and  changing  conditions,  especi- 
ally with  the  possible  mandatory  adoption 
of  the  criteria  in  regulatory  processes. 
It  is  strongly  recommended  that  similar 
consideration  be  given  by  others  in  gener- 
ating specific  criteria  since  such  systems 
may  not  provide  for  detailed  understanding 
and  periodic  reviews  required  under  the 
ASTM  voluntary  consensus  system. 

Likewise  it  is  expected  this  document 
will  find  good  application  in  normal  survey 
and  audit  activities  for  quality  assurance 
programs  and  in  other  technical  management 
areas . 

Similar  applications  may  also  be  found 
in  the  growing  requirements  for  current  good 
laboratory  practice,  current  good  manufac- 
turing practice,  and  in  reasonable  testing 
programs . 


The  intent  of  this  standard  has  been  to 
provide  another  viable  tool  in  the  process 
of  applying  standards  in  improving  the  pro- 
ductivity of  the  world.     To  this  end,  Tech- 
nical Committee  E-36  extends  a  cordial  in- 
vitation for  all  interested  to  participate 
in  the  ongoing  work  of  the  Committee  and 
in  developing  new  projects  which  might  fall 
within  the  scope  of  the  Committee. 
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This  paper  describes  the  principal  services  provided  by  the  U.S. 
National  Bureau  of  Standards  that  allow  laboratories  to  assess  and 
maintain  at  a  high  level  the  quality  of  their  measurements.     The  NBS 
programs  described  in  some  detail  in  this  paper  include:  Standard 
Reference  Materials,  Calibration  Services,  Measurement  Assurance 
Program  Services,  and  Collaborative  Reference  Programs.     The  paper 
deals  with  the  basic  aims  and  objectives  of  these  services.     A  brief 
description  of  each  service  is  provided,  along  with  indications  of  how 
these  services  have  improved  the  performance  of  laboratories.  Also 
discussed  are  general  concepts  regarding  the  assurance  of  measurement 
accuracy  and  compatibility  among  laboratories  on  a  national  and/or 
international  scale. 

Keywords:     Calibrations;  Collaborative  Reference  Programs;  laboratory 
performance;  measurement  assurance;  measurement  evaluation  technology; 
measurement  services,  physical  standards;  proficiency  testing;  Standard 
Reference  Materials;  testing;  traceability . 


1.  Introduction 

Our  society  today  relies  on  accurate 
measurements  for  realizing  equity  in  the 
marketplace,  for  industrial  quality  control, 
for  equitable  enforcement  of  and  compliance 
with  government  regulations  related  to  the 
environment,  health  and  safety,  and  for 
scientific  research.     In  order  that  the  un- 
certainty of  measurements  be  sufficiently 
small  so  that  correct  decisions  can  be  made 
based  on  measured  data,  measurement  quality 
assurance  must  be  built  into  the  measurement 
system. 

There  is  an  increasing  demand  for  the 
accreditation  of  laboratories  that  are  cap- 
able of  performing  reliable  measurement  ser- 
vices in  all  areas  of  the  measurement  field. 
Accredited  laboratories  and  the  accrediting 
authority  must  be  particularly  sensitive  to 
the  need  for  measurement  quality  assurance. 
Calibration  and  standardization  of  labora- 
tory equipment  and  procedures  are  essential 


to  obtain  the  reliable  measurements  neces- 
sary for  modern  technology  and  to  assure 
that,  for  fair  trade  and  commerce,  credible 
testing  laboratories  can  be  identified  and 
called  on  to  determine  if  the  quality  of  the 
product  adequately  reflects  the  specifica- 
tion requirements.     Table  1  lists  the  prin- 
cipal factors  that  a  laboratory  should  con- 
sider incorporating  into  an  overall  system 
of  quality  assurance. 

A  key  responsibility  of  the  National 
Bureau  of  Standards  (NBS),  as  stated  in  the 
act  of  Congress  that  establishes  NBS  and 
subsequent  amendments  to  that  act,  is  to 
establish  and  maintain  national  standards 
of  measurement,  coordinate  these  standards 
with  those  of  other  nations,  and  provide 
means  and  methods  for  making  measurements 
consistent  with  those  standards.     In  other 
words,  provide  a  complete  and  consistent 
national  system  of  physical  measurements. 


"Agreed  upon"  standards  are  necessary 
for  making  accurate  measurements.     As  a  fa- 
miliar example,  consider  the  prototype  mass 
standards  Kh  and  K20  kept  by  NBS  which  are 
the  U.S.   standards  for  the  kilogram.  By 
mutual  agreement  of  the  measurement  commun- 
ity, mass  measurements  in  the  U.S.  are  ulti- 
mately referred  to  these  national  standards 
of  mass. 

One  often  hears  of  requirements  to  the 
effect  that  "measurements  or  a  reference 
standard  must  be  traceable  to  the  National 
Bureau  of  Standards".     Presumably  the  intent 
of  such  traceability  citations  is  to  insure 
that  measurements  are  sufficiently  accurate 
that  proper  decisions  can  be  made.     In  a  re- 
cent article,  Belanger  has  described  the  con- 
fusion that  can  result  when  differing  inter- 
pretations of  traceability  are  adopted  by 
various  parties^!].     If  it  is  required  that 
the  measurements  made  by  an  accredited  lab- 
oratory be  "traceable"  to  NBS,  then  it  is 
clear  that  the  laboratory,  the  accrediting 
party,  and  the  customer  must  agree  on  the 
definition  of  traceability  and  on  the  cri- 
teria as  to  what  constitutes  adequate  trace- 
ability. 

NBS  prefers  a  definition  of  traceability 
that  focuses  on  the  uncertainty  of  the  mea- 
surements being  performed.!    It  is  clear  that 
NBS  must  provide  convenient  mechanisms  by 
which  parties  outside  NBS  can  verify  that 
their  measurements  are  accurate  at  some  de- 
sired level  of  uncertainty  relative  to  na- 
tional standards.     Such  mechanisms  are  need- 
ed whether  or  not  one  has  accredited  labor- 
atories. 

Laboratories  that  must  make  quality 
measurements  consistent  with  national  stan- 
dards have  available  a  wide  variety  of  NBS 
measurement  services  from  which  to  choose. 
Table  2  shows  the  measurement  transfer 


1  In  referenced!  traceability  is  defined 
as  follows:     "Traceability  to  designated 
standards  (national,  international,  or  well- 
characterized  reference  standards  based  upon 
fundamental  constants  of  nature)  is  an  attri- 
bute of  some  measurements.  Measurements 
have  traceability  to  the  designated  standards 
if  an  only  if  scientifically  rigorous  evi- 
dence is  produced  on  a  continuing  basis  to 
show  that  the  measurement  process  is  produc- 
ing measurement  results  (data)  for  which  the 
total  measurement  uncertainty  relative  to 
national  or  other  designated  standards  is 
quantified. " 
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mechanisms  employed  by  NBS.     This  paper  fo- 
cuses on  those  services  of  NBS  that  direct-  " 
ly  facilitate  the  making  of  accurate  mea- 
surements by  laboratories  throughout  the  U.Ei 

2.     Calibration  Services 

ll 

J  e 

The  calibration  of  working  standards  is] 
a  service  that  has  been  offered  to  the  pub- 
lic by  NBS  ever  since  it  was  formed.  Today, 
in  a  typical  year,  NBS  provides  a  total  of 
several  thousand  calibrations  for  approxi- 
mately 1000  different  organizations  -  in- 
cluding industrial  standards  laboratories, 
Department  of  Defense  and  other  Federal 
agency  calibration  activities,  state  and 
local  weights  and  measures  activities,  uni- 
versities, hospitals,  utility  companies, 
etc.     NBS  currently  offers  calibration  ser-  s 
vices  for  mass,  volume,  density,  temperature 
pressure,  dc  and  low  frequency  electrical 
measurements,  rf  and  microwave  measurements,  i ' 
radiation,  and  a  variety  of  other  measure- 
ments.    The  array  of  services  is  constantly 
changing  since  each  year  NBS  generally  ini- f 
tiates  calibration  services  to  reflect  new 
needs  associated  with  emerging  technologies. 
Also,  to  make  way  for  new  services,  NBS  reg- 
ularly phases  out  services  that  have  become 
less  important  to  the  measurement  community 
because  of  changing  technology,  services  of- 
fered by  the  private  sector,  or  declining 
requirements . 

It  is  clear  that  NBS  does  not  have  the 
personnel  to  calibrate  all  standards  in  the 
U.S.  of  a  given  type  that  require  calibra-  ; 
tion  nor  would  it  be  possible  for  us  to  of-  1 
fer  high  level  calibration  services  for  all  1 
conceivable  measurement  quantities  for  which; 
accurate  measurements  are  needed.  Accord- 
ingly, NBS  tries  to  provide  a  mix  of  ser-  E 
vices  that  is  responsive  to  the  most  impor- 
tant needs  of  the  measurement  community  and 
where  possible  to  provide  those  services  to 
high  level  laboratories  that  can  in  turn 
provide  calibration  services  and  thereby 
transfer  accuracy  to  lower  level  field  mea- 
surements.    It  is  the  policy  of  the  Federal 
government  not  to  compete  with  private  sec-  ' 
tor  services.     Accordingly,  NBS'  services  r 
are  limited  to  those  special  high  level  cali 
brations  which  are  not  readily  available 
elsewhere  and  high  accuracy  calibrations  re-; 
quired  for  direct  traceability  to  national 
standards . 

NBS  publishes  a  catalog  of  calibration 
services,  NBS  Special  Publication  250, 
"Calibration  and  Related  Measurement  Ser- 
vices of  the  National  Bureau  of  Standards". 
An  appendix  to  SP  250  is  issued  each  June 
and  December  giving  up-to-date  prices  and 


information  on  new  services  that  are  about 
to  be  offered  or  services  that  are  slated 
for  phase-out.     Both  of  these  documents  are 
available  from  the  Office  of  Measurement 
Services  at  NBS. 

Laboratories,  whether  formally  accred- 
ited or  not,  wishing  to  maintain  a  high  lev- 
el of  quality  control  over  their  measurements 
can  effectively  utilize  NBS  calibration  ser- 
vices, and  many  do  so.     In  recent  years,  NBS 
and  its  calibration  customers  have  come  to 
recognize  that  the  use  of  standards  cali- 
brated by  NBS  is  by  itself  no  guarantee  that 
the  laboratory  is  performing  adequate  mea- 
surements.    Even  if  a  particular  lab  has  a 
\  standard  accurately  calibrated  by  NBS,  it 
probably  will  not  be  able  to  make  accurate 
measurements  unless  the  operators  are 
skilled,  the  environment  is  adequately  con- 
trolled and  the  measurement  procedures  are 
technically  sound.     Recognizing  this  fact, 
laboratory  accreditation  procedures  gener- 
ally address  all  of  these  factors. 

In  the  section  which  follows,  special 
NBS  services  that  allow  our  customers  to 
<  control  their  total  measurement  process  are 
described  [2]. 

3.     Measurement  Assurance  Programs 

NBS  has  developed  special  services 
called  Measurement  Assurance  Program  (MAP) 
Services  to  aid  laboratories  in  establishing 
their  measurement  uncertainty  and  maintain- 
ing it  at  an  acceptable  level.     NBS  current- 
ly provides  these  services  for  measurements 
of  mass,  temperature  (platinum  resistance 
thermometers),  dc  voltage,  dc  voltage  ratio, 
capacitance,  resistance,  electric  energy 
(watt-hour  meters),  laser  power  and  energy, 
and,  on  a  trial  basis,  gage  blocks. 

The  techniques  used  in  MAP's  are  famil- 
iar techniques  to  those  experienced  in  the 
j    quality  assurance  field.     Unlike  a  calibra- 
j  tion  which  only  checks  the  standard  or  in- 
strument, the  MAP  is  designed  to  assess  the 
performance  of  the  entire  measurement  pro- 
i   cess.     A  key  element  of  MAP  is  the  availa- 
bility of  a  suitable  transport  standard. 
The  MAP  transport  standard  is  a  device,  ar- 
tifact, or  material ^  that  is  stable,  rugged, 
and  well-characterized,  and  whose  value  is 
accurately  known  relative  to  national  stan- 
dards at  NBS.     This  standard  is  sent  to  the 


Standard  Reference  Materials  can  be  used 
as  the  basis  for  a  MAP  when  coupled  with 
appropriate  methods  and  data  analysis. 


participating  laboratories  where  it  is  mea- 
sured as  an  unknown  or  "blind"  sample.  Since 
NBS  measures  the  item  before  it  leaves  the 
Bureau  and  after  it  returns,  its  value  while 
in  the  participant's  laboratory  should  be 
accurately  known.     Any  bias  or  offset  (sys- 
tematic error)  associated  with  the  measure- 
ments made  by  the  MAP  participant  can  be  as- 
certained from  the  data  returned  to  NBS. 

Since  an  objective  of  the  MAP  is  to 
establish  the  total  uncertainty  of  the  mea- 
surement process  in  the  participating  lab- 
oratory, it  is  also  necessary  for  the  lab- 
oratory to  have  suitable  working  standards 
upon  which  measurements  can  be  made  at  ap- 
propriate intervals  to  establish  the  pro- 
cess precision  and  insure  that  it  remains 
within  acceptable  limits.     NBS  provides  ad- 
vice on  applicable  quality  control  tech- 
niques such  as  the  keeping  of  control  charts 
to  monitor  the  random  error  (precision)  of 
the  process  and  make  sure  that  the  process 
remains  in  a  state  of  statistical  control. 
Knowing  both  the  systematic  error  or  bias 
by  means  of  data  taken  on  the  NBS  transport 
standard  and  the  random  error  established 
by  the  measurements  made  in  the  laboratory 
over  a  suitable  period  of  time, it  is  possi- 
ble for  NBS  to  issue  a  test  report  that 
quantifies  the  total  measurement  uncertainty 
of  the  participant.     NBS  provides  consulting 
help  to  new  MAP  participants  in  identifying 
problems  and  generally  insuring  that  their 
accuracy  objectives  are  met. 

NBS  disseminates  measurement  assurance 
services  in  two  ways.     Most  users  deal  with 
NBS  on  a  one-on-one  basis,  but  NBS  and  our 
customers  are  increasingly  interested  in 
using  a  group  or  regional  approach.  This 
second  approach^]  has  been  particularly  suc- 
cessful in  the  case  of  dc  voltage,  and  so 
NBS  is  looking  for  opportunities  to  utilize 
the  same  approach  for  other  quantities.  The 
regional  or  group  MAP  involves  a  group  of 
laboratories  in  one  part  of  the  country 
(e.g.,  Southern  California)  interacting  with 
NBS  as  a  group  through  one  laboratory  desig- 
nated as  the  "pivot"  laboratory.     There  ap- 
pear to  be  a  number  of  advantages  both  to 
NBS  and  to  our  customers  from  the  use  of  the 
more  efficient  group  or  regional  approach, 
including  lower  costs  to  the  participants, 
faster  turnaround  time,  and  in  the  long  run, 
less  frequent  need  for  interacting  with  NBS 
to  verify  measurement  accuracy. 

The  net  result  of  a  MAP  is  that  a  rig- 
orous uncertainty  statement  can  be  devel- 
oped for  the  performance  of  the  measurement 
system.     Additionally,  it  provides  a  mech- 
anism for  noting  changes  with  time  or  out- 
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of-control  conditions.     Should  some  organi- 
zation, either  private  or  governmental,  wish 
to  institute  a  laboratory  accrediation  pro- 
gram for  calibration  laboratories,  data  from 
a  MAP  (assuming  an  NBS  service  is  available 
for  the  measurement  quantity  of  interest) 
can  provide  an  excellent  tool  for  assessing 
laboratory  performance.     Note  that  the  MAP 
approach  is  a  "systems"  type  approach  and 
that  it  has  much  in  common  with  the  philos- 
ophy of  the  Standard  Reference  Materials 
Program  as  described  in  the  section  which 
follows . 

h.     Standard  Reference  Materials 

Standard  Reference  Materials  (SRM's) 
have  been  produced,  certified  and  issued  by 
NBS  since  1950 W .     SRM's  are  well-charac- 
terized, homogeneous,  stable  materials  (or 
simple  artifacts)  with  specific  properties 
measured  and  certified  by  NBS.     Over  1,000 
different  SRM's  are  now  available  from  NBS 
[5].     The  SRM  inventory  can  be  categorized 
into  three  basic  types  of  certified  refer- 
ence materials.     These  categories  are:  l) 
materials  certified  for  chemical  composition 
or  purity,  2)  materials  certified  for  one 
or  more  fundamental  physical  or  chemical 
properties  such  as  pH  or  melting  point  and 
3)  materials  or  artifacts  certified  for  other 
properties  usually  associated  with  special 
engineering  type  applications,  e.g.,  sieve 
sizing  or  color  fading.     Many  of  the  SRM's 
in  this  latter  category  are  certified  on  a 
method-dependent  rather  than  absolute  basis. 
SRM's  are  widely  used  throughout  the  world 
in  a  variety  of  measurement  applications  in- 
cluding development  and  evaluation  of  mea- 
surement methods,  assurance  of  long-term 
measurement  compatibility  among  different 
laboratories  and  establishment  of  measure- 
ment traceability  to  NBS. 

During  fiscal  year  1978,  NBS  distrib- 
uted 37,000  SRM  units  to  over  10,000  users 
throughout  the  world.     Foreign  laboratories 
represent  over  20  percent  of  SRM  users.  The 
four  major  SRM  user  categories  are:     l)  in- 
dustrial research  and  quality  control  lab- 
oratories [61,  2)  environmental  analysis  and 
monitoring  laboratories C 7] ,  3)  laboratories 
performing  clinical  or  health  related  mea- 
surementsL 8,9,10]  an(}  1^)  laboratories  en- 
gaged in  basic  scientific  and/or  metrologi- 
cal  research  (e.g.,  establishment  of  an  ac- 
curate temperature  scale)  [H 5 12]  , 

One  of  the  most  significant  outputs  of 
the  NBS  SRM  program  during  the  last  10  years, 
has  been  the  development  of  a  greater  under- 
standing of  the  role  of  SRM's  in  total  mea- 
surement systems.     The  NBS  Office  of  Standard 


Reference  Materials  (0SRM)  has  placed  an  ii 
creased  emphasis  on  utilizing  a  "systems 
approach"  to  accurate  measurement,  whereby 
NBS  SRM's  together  with  standardized  well- 
characterized  test  methods  (reference  meth 
ods )  are  used  to  transfer  accuracy  and  es- 
tablish measurement  traceability  throughout 
large  multi-laboratory  measurement  networks 
As  indicated  earlier,  the  systems  approach 
in  the  SRM  Program  shares  many  common  fea 
tures  with  MAP  activities.     A  large  number 
of  articles  and  monographs  have  been  pub- 
lished by  J. P.  Call  [13-15]  and  other  mem- 
bers of  the  OSRM  staff  [l6-l8,6,7,8]  con- 
cerning the  systems  approach  to  accurate 
and  compatible  measurements. 

The  systems  approach  to  measurement 
compatibility  recognizes  the  fact  that  SRM': 
provide  a  highly  useful  and  important,  but 
by  no  means  sufficient  mechanism  for  achie^ 
ing  measurement  compatibility  on  a  nationa] 
or  international  scale.  Measurement  compal 
ibility  means  that  all  measurement  stations 
within  a  given  measurement  network  obtain 
values  within  agreed-upon  measurement  un- 
certainties, when  measuring  a  specific  pro] 
erty  of  the  same  material.  The  systems  ap- 
proach to  measurement  compatibility  is 
illustrated  in  Figure  1  and  described  ex- 
tensively in  the  review  paper  by  Uriano  anc 
Gravatt[l6] . 


Figure  1  represents  a  hierarchical  sys 
tern  of  analytical  methods  and  reference  ma< 
terials  each  coupled  to  the  other  in  the 
manner  shown.     The  function  of  each  compo- 
nent ( I  to  VI)  is  to  transfer  accuracy  to 
the  level  immediately  below  it  and  to  help  jj 
provide  traceability  to  the  level  immedi- 
ately above  it,  thus  helping  to  assure  mea 
surement  compatibility  in  the  overall  systtf 

As  we  proceed  from  bottom  to  the  top 
the  measurement  hierarchy,  accuracy  require 
ments  increase  at  the  expense  of  decreased 
measurement  efficiency.     At  the  top  of  the 
hierarchy  are  the  so-called  definitive  metl 
ods  of  analysis  or  test,  which  give  the  moE 
accurate  values  obtainable. 3  Unfortunately 
most  definitive  methods  (e.g.,  gravimetric 
techniques  for  preparing  analyzed  gas  SRM'e 
[l6])  are  usually  time  consuming,  sophisti- 
cated methods  and  thus  are  not  economically 
acceptable  for  widespread  and  routine  field 
use.     Definitive  methods,  however,  are  usee? 


3  For  definitions  and  more  detailed  discus> 
sion  of  definitive,  reference  and  field 
methods,  see  ref 


1.  ren 
[l6]. 
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whenever  possible  to  certify  NBS  SRM's. 
SRM's  then  can  he  used  to  transfer  accuracy 
further  down  the  measurement  hierarchy,  e.g., 
to  verify  the  accuracy  of  or  calibrate  ref- 
erence methods. 
i 

Reference  methods,  which  may  be  auto- 
mated comparative  methods  (e.g.,  nondisper- 
.jsive  infrared  spectrometry  for  measuring 
carbon  monoxide  in  air)  can  then  be  used  di- 
rectly in  the  field  or  alternatively  as  a 
•basis  for  developing  or  evaluating  other 
methods.     Reference  methods  are  also  com- 
monly used  for  producing  secondary  reference 
materials,  which  in  turn  are  directly  used 
in  routine  field  measurement  applications. 
Examples  of  reference  methods  are  the  Envi- 
ronmental Protection  Agency  reference  meth- 
ods for  monitoring  the  ambient  air  criteria 
•■pollutants [20].     The  Annual  Book  of  ASTM 
Standards  lists  hundreds  of  standard  test 
•.methods,  which  can  be  designated  as  refer- 
ence (or  in  some  cases  definitive)  methods. 
For  example,  the  ASTM  E-350  series  of  Stan- 
dard Methods  for  chemical  analysis  of  a 
:■  variety  of  iron  and  steel  products  [21  ]  con- 
.  stitutes  an  excellent  example  of  reference 
methods [l6].     In  fact,  the  systems  approach 
to  compatible  measurement  historically 
:;ioriginated  in  the  metals  industries  [  ^5 13] , 
where  NBS  SRM's  and  ASTM  standard  methods 
both  play  a  critical  role  in  assuring  mea- 
surement compatibility. 

The  accurance  of  numerous  field  methods 
■can  in  principle  be  traced  back  to  a  defini- 
tive method  in  such  a  hierarchical  accuracy- 
based  measurement  system.     In  such  a  system, 
SRM's  and  other  reference  materials  play  a 
•.■necessary,  role  in  the  transfer  of  accuracy. 
However,  SRM's,  by  themselves,  cannot  assure 
[ high  quality  measurements.     As  in  the  case 
, of  MAP's,  good  methods,  good  laboratory 
-practices,  well-qualified  personnel  as  well 
as  proper  intra-laboratory  (internal)  and 
■inter-laboratory  (external)  quality  assur- 
ance procedures  are  just  as  important  as  the 
reference  materials. 

Variations  of  the  idealized  accuracy- 
based  measurement  system  are  now  in  place  in 
•  many  industrial  areas  and  are  being  imple- 
mented in  the  areas  of  clinical  and  environ- 
mental analysis [ 7 5 10 ] .     Many  other  areas  are 
tjalso  beginning  to  utilize  this  approach.  An 
excellent  illustrative  example  of  the  systems 
approach  to  measurement  compatibility,  which 
^involves  both  SRM's  and  ASTM  standard  meth- 
.  ods,  is  represented  by  the  recent  Department 
of  Commerce  National  Voluntary  Laboratory 
Accreditation  Program  (NVLAP)  for  testing 
the  properties  of  thermal  insulation  materi- 
als^].    This  is  the  first  NVLAP  program 


being  implemented  by  the  Secretary  of  Com- 
merce.    A  number  of  NBS  SRM's  and  ASTM 
Standard  Methods  are  cited  in  the  criteria 
[22]  for  accrediting  laboratories,  covering 
such  properties  as  particle  size  and  thermal 
resistance . 

Figure  2  schematically  illustrates  the 
thermal  resistance  measurement  system.  NBS 
is  in  the  process  of  developing  a  series  of 
high  and  low  density  fiberglass  SRM's^  cer- 
tified for  thermal  resistance  (R-values)  as 
a  function  of  temperature  and  density.  The 
NBS  SRM's  are  certified  by  a  definitive  meth- 
od (The  NBS  Guarded  Hot  Plate  Method).  Hav- 
ing been  certified,  the  SRM's  can  then  be 
used  to  control  the  accuracy  of  reference 
methods  such  as  the  ASTM  Guarded  Hot  Plate 
Method  (ASTM  C1TT)  or  for  direct  calibration 
of  comparative  reference  methods  such  as  the 
Heat  Flow  Meter  Method  (ASTM  C5l8).  Thus 
the  NBS  SRM's  and  ASTM  methods  are  intended 
for  direct,  continuous  use  in  intra-labora- 
tory quality  assurance  programs. 

NBS  is  also  developing  a  set  of  thermal 
insulation  test  specimens  for  use  in  profi- 
ciency testing,  i.e.,  an  external  or  inter- 
laboratoty  quality  assurance  program.  This 
will  provide  a  mechanism  for  directly  moni- 
toring (auditing)  the  performance  of  parti- 
cipating laboratories  on  a  periodic  basis. 
The  audit  program  provides  the  measurement 
system  with  an  appropriate  feedback  and 
traceability  mechanism. 

Table  3  contains  a  summary  of  the  most 
common  uses  of  SRM's  in  measurement  appli- 
cations.    The  reader  is  referred  to  various 
papers  cited  previously [H ,13-17]  for  other 
specific  examples  of  the  use  of  SRM's.  The 
review  paper [l6]  cited  previously  also  dis- 
cusses the  relative  roles  of  various  organ- 
izational components  (e.g.,  voluntary  stan- 
dards organizations,  national  standards 
laboratories,  regulatory  agencies)  in  assur- 
ing measurement  compatibility. 

5.     Collaborative  Reference  Programs 

NBS  has  been  involved  for  many  years 
in  programs  for  proficiency  assessment  of 
laboratories  in  cooperation  with  private 
industry  and  standards  organizations.  These 
programs  were  initiated  to  assist  laborato- 
ries in  determining  the  accuracy  level  and 


k  The  first  of  these  SRM's,   (SRM  1^50  -  High 
Density  Fiberglass)  is  now  available  from 
NBS  1 5 ] . 
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precision  of  their  testing  and  in  improving 
their  testing  performance.     The  programs  are 
sponsored  by  a  variety  of  organizations  and 
are  conducted  under  a  variety  of  procedures. 
Each  field  of  testing  has  its  special  prob- 
lems and  each  sponsoring  organization  its 
own  goals.     Hence,  no  two  of  the  existing 
programs  are  identical,  but  each  has  been 
designed  to  meet  the  specific  needs. 

In  1936,  the  first  collaborative  refer- 
ence sample  program  was  initiated  for  cement 
as  part  of  the  NBS  Research  Associate  Program 
of  the  Cement  Reference  Laboratory  of  the 
American  Society  for  Testing  and  Materials 
(ASTM)  Committee  C-l.     In  1966,  this  program 
was  revised  to  essentially  its  present  form 
wherein  two  pairs  of  samples  for  physical 
tests  and  two  pairs  of  samples  for  chemical 
analyses  are  distributed  each  year.  Similar 
programs  for  bituminous,  soils,  aggregates, 
and  portland  cement  concrete  and  bituminous 
concrete  were  then  established  in  rapid  suc- 
cession, the  last  in  197^ •     These  programs 
are  under  the  sponsorships  of  ASTM  and  the 
American  Association  of  State  Highway  and 
Transportation  Officials,  respectively. 

In  1969  a  bimonthly  program  for  paper 
and  board  testing  was  initiated  under  the 
sponsorship  of  the  Technical  Association  of 
the  Pulp  and  Paper  Industry  and  a  program 
for  control  of  the  quality  of  shipping  con- 
tainer components  was  established  for  the 
Fourdrinier  Kraftboard  Institute.     This  later 
program  involves  weekly  testing  and  monthly 
reports  by  the  participants.     A  quarterly 
program  for  the  rubber  industry  was  started 
in  1970  with  the  help  of  ASTM  Committee 
D-ll,  and  a  color  and  appearance  program, 
of  interest  to  many  industries  and  users, 
was  established  for  the  Manufacturers  Coun- 
cil on  Color  and  Appearance.     This  quarterly 
program  presently  includes  tests  for  gloss, 
color  and  color  difference,  and  retroreflec- 
tivity . 

The  Collaborative  Reference  Program 
(CRP)  provides  a  participating  laboratory 
with  a  means  for  checking  periodically  the 
level  and  uniformity  of  its  testing  perfor- 
mance in  comparison  with  that  of  other  lab- 
oratories.    A  major  feature  of  all  CRPs  is 
the  periodic  distribution  to  the  participat- 
ing laboratories  of  suitably  selected  and 
prepared  samples.     The  participants  are  in- 
structed to  test  the  samples  in  accordance 
with  standard  test  methods  and  supplemental 
instruction  provided  by  NBS.     They  are  also 
requested  to  report  any  deviations  from  the 
standard  procedure,  as  for  example,  when  the 
conditions  in  the  test  room  are  not  strictly 
in  compliance  with  the  requirements  of  the 


standard.  Special  data  forms  are  provided 
to  the  laboratories  for  reporting  results  1 
to  NBS. 

The  test  results  are  analyzed  at  NBS 
accordance  with  the  Youden  two-sample  pro- 
cedureL'23]  using  special  computer  programs 
The  results  of  the  analysis  are  presented 
using  tables  and  Youden' s  graphical  tech- 
nique, as  shown  in  Figure  3.     In  this  dia- 
gram, each  point  represents  a  laboratory 
and  the  ellipse  represents  the  bounds  with 
which  95  percent  of  the  points  for  similar 
laboratories  are  located. 

j 

6.  Technical  Support  for  the  Nationa: 
Voluntary  Laboratory  Accreditation 
Program 

'  j 

In  February  1976,  the  Department  of 
Commerce  (DoC)  established  the  National 
Voluntary  Laboratory  Accreditation  Program 
(NVLAP).  The  purpose  of  this  program  is  t 
give  recognition,  through  accreditation,  t 
laboratories  that  have  the  capabilities  to 
correctly  perform  specific  tests  according 
to  identified  standard  methods.  There  are 
three  (3)  different  procedures  for  institu 
ing  laboratory  accreditation  programs  unde 
NCLAP.  Procedure  (A)  is  for  the  general 
public  and  it  requires  that  four  distinct 
phases  be  accomplished,  i.e.,  finding  of 
need,  establishment  of  a  criteria  committe 
development  of  criteria,  and  the  examinati 
for  accreditation.  Procedures  (B)  and  (C) 
are  for  the  use  of  Federal  agencies  and  pr 
vate  consensus  organizations  (consensus  by 
DoC  definition5)  respectively.  Both  (B)  a 
(C)  permit  by-passing  the  requirements  for 
"finding  of  need"  and  the  establishment  of 
a  criteria  committee.  NBS  provides  techni 
cal  support  to  this  program  by  assisting  ij 
the  development  of  both  general  and  specif! 
criteria  and  by  evaluating  the  capabilitie 
of  laboratories  that  request  accreditation 
[2U,25].  The  examination  methodology  cove 
three  phases:  l)  Questionnaire,  2)  On- sit 
inspection;  and  3)  Proficiency  testing. 

For  proficiency  testing  the  "true"  or:- 
target  test  result  for  any  particular  test 
is  obtained  by  one  of  the  following  ways :  I 

(l)  Manufacturer .  For  some  propertie 
of  a  sample,  it  is  possible  to  determ 


■1 


5  Section  7c. 3(b)  of  Federal  Register, 
Volume  hk,  Number  8l,  April  25,  1979,  Page1' 
21+27^-2^282  (15  CFR  Part  7c)  lists  six  err 
teria  that  "private  sector  organizations"  > 
must  meet  in  order  to  be  in  this  category. 
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what  the  test  result  should  be  from  in- 
formation on  how  the  sample  was  made. 
However,  each  case  has  to  be  thoroughly 
examined  before  manufacturing  informa- 

.  tion  can  be  used  as  the  basis  for  de- 
termining the  target  or  "true"  values. 
This  approach  is  often  useful  in  profi- 
ciency testing  programs  requiring  qual- 

.   itative  responses  or  identifications 
only  (e.g.,  is  starch  present  or  not). 

f 

(2)  Reference  Laboratory.     Sometimes  a 
single  laboratory,  such  as  the  National 
Bureau  of  Standards,  has  sufficiently 
:   high  competence  and  national  recogni- 
:   tion  that  it  can  be  used  to  provide  the 
target  or  "true"  test  result.     This  is 
particularly  useful  when  the  laboratory 
has  the  capability  of  and  has  agreed  to 
carefully  verify  the  correctness  of 
i  every  important  dimension  of  its  appa- 
!  ratus  and  every  step  in  its  application 
j  of  the  standard  test  method. 
j( 

i   (3)  Group  of  Reference  Laboratories. 
=  When  no  single  laboratory  can  be  given 
a  national  recognition  as  having  suffici- 
J  ently  high  competence  to  set  the  nation- 
al standard,  it  sometimes  is  possible 
for  a  proficiency  test  coordinator  to 
use  the  results  from  a  number  of  rep- 
I  utable  laboratories.     This  would  be  ac- 
i  complished  by  pooling  their  results 
H  (after  a  suitable  statistical  check  on 
«  the  agreement  among  the  results)  in 
[  order  to  establish  the  target  or  "true" 
!  test  result. 

:    (1+)  Reference  Method.     Under  NVLAP  pro- 
|  cedures,  the  standard  test  method  will 
a  usually  also  be  the  reference  method. 
r:i  However,  in  some  cases  the  standard  test 
a  method  may  be  so  broadly  written  as  to 
y  permit  a  wide  variety  of  test  equipment 
9  and  testing  protocols.     If  in  such  a 
[  case  a  particular  protocol  and  equipment 
J  combination  is  recognized  as  a  reference 
'  method  (or  can  be  shown  through  error 

analysis  to  yield  results  well  within 
i  the  required  precision  and  accuracy), 

then  the  results  obtained  with  that 

method  by  one  or  more  "reference"  lab- 
i   oratories  are  used  to  establish  the 

target  test  result. 

(5)  Participants .     If  there  is  a  suf- 
ficient number  of  participating  test- 
ing laboratories  and  an  insufficient 
'f,  number  of  reference  laboratories,  then 
r  the  test  results  of  the  participating 

laboratories  are  sometimes  pooled  by  a 
I  proficiency  test  coordinator  to  estab- 
lish the  target  result  for  the  individ- 
ual participants.     It  is  important  j 


that  the  pooled  test  results  include 
only  test  data  from  laboratories  known 
(on  the  basis  of  all  available  infor- 
mation) to  be  following  the  standard 
test  method.     This  is  determined  not 
from  the  test  data,  but  from  an  inspec- 
tion report  and  from  information  sub- 
mitted originally  and  with  the  test  da- 
ta. 

( 6 )  Previous  Proficiency  Test  and 
Inter-laboratory  Data.     Sometimes  the 
same  samples  are  used  as  were  used  in  a 
previous  proficiency  test.     If  so,  the 
new  target  test  result  is  based  on  a 
weighted  pooling  of  current  and  previ- 
ous test  results. 

7 .  Summary 

The  recognition  of  the  need  for  a  con- 
sistent measurement  system  led  to  the  forma- 
tion of  the  National  Bureau  of  Standards  and 
in  particular  NBS1  responsibility  for  "The 
custody,  maintenance,  and  development  of  the 
national  standards  of  measurement,  and  the 
provision  of  means  and  methods  for  making 
measurements  consistent  with  those  stan- 
dards."    The  laboratory  performance  evalua- 
tion services  of  NBS  benefit  all  segments  of 
our  society  through  the  promotion  of  quality 
measurements  relative  to  national  standards 
whether  the  concern  is  for  fair  trade  in  the 
marketplace  or  determining  the  pull  of  grav- 
ity in  a  scientific  investigation. 
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1.     What  is  a  Measuring  Process? 

A  measuring  process  involves  a  rela- 
tionship between  the  quantity  of  some 
property  of  a  material,  product  or  system 
and  an  indicating  device  sensitive  to  that 
property.     A  simple  familiar  example  is  the 
thermometer  which  indicates  the  temperature 
of  the  surrounding  material  by  the  expansion 
of  mercury  in  glass.     Another  example  is  the 
voltmeter  whereby  the  voltage  is  indicated 
by  the  angular  deflection  of  the  needle. 

Ideally  there  should  be  a  stable 
linear  relationship  between  the  quantity 
being  measured  and  the  response  of  the 
indicating  device.     However,  even  in  such  a 
simple  example  as  the  mercury- in-glass 
thermometer  the  relationship  is  not  exactly 
linear  nor  is  it  stable  especially  before 
1   the  glass  has  been  adequately  "aged."  Non- 
linearity,  however,  is  not  a  serious  problem 
if  the  device  can  be  calibrated  simply  or  a 
simple  theory  relates  measurement  and  quan- 
tity.    Examples  of  non-linear  relationships 
are  ohmeter  needle  deflection  vs  electrical 
resistance  and  transmittance  as  a  measure  of 
concentration. 


Generally,  the  relationship  between  the 
measurement  and  the  quantity  being  measured 
may  be  represented  graphically  by  a  mono- 
tonically  increasing  or  decreasing  curve 
(Figure  1). 


Q  Q 


Figure  1.     Relationship  Between  Measurement 
(M)  and  Quantity  (Q)  Being  Measured 
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One  essential  requirement  is  that  the 
relationship  between  measurement  and  quantity 
be  monotonic.     The  situation  shown  in 
Figure  2  is  unacceptable  because  of  its 
ambiguity:     the  measurement  M  could  mean 
that  the  quantity  is  either  Q    or  Q^. 


ignoring  sample  variability  and  other  sample 
problems,  measurement  of  the  samples  will 
result  in  a  curve  relating  the  measurements 
to  the  quantity  being  measured  (Figure  3) . 


Figure  2.     Ambiguous  Relationship:  Does 
Measurement   (M)  Indicate    Quantity  Q1  or  Q^i 


A  measuring  process  usually  is  defined 
by  a  protocol  or  standard  test  method  such 
as  an  ASTM  standard.     In  addition  to  sampling 
error,  a  realization  of  a  measuring  process 
involves  four  elements: 

(1)  The  clarity  and  specificity  of 
the  test  protocol; 

(2)  An  observer  or  operator  and  his 
or  her  interpretation  of  the 
protocol; 

(3)  The  type,  condition,  and 
calibration  of  the  instrument 
and  equipment  used; 

(4)  The  environment  (temperature, 
humidity,  air  polution,  air 
movement,  atmosphere  pressure, 
vibration,  gravity,  intensity 
and  wave-length  distribution  of 
illumination,  etc.)- 

Consider  a  single  laboratory  and  for  the 
moment  one  operator  and  one  set  of  test  equip- 
ment.    Over  a  short  period  of  time  the 
environment  will  usually  remain  reasonably 
constant.     Then  for  a  series  of  samples 
covering  the  test  range  of  interest  and 
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Figure  3.     Monotonic  Relationship  Between 
Measurement  (M)  and  Quantity  (Q) 


If  a  second  and  a  third  series  of 
samples  identical  to  the  first  series  are 
measured  in  the  same  laboratory  but  by 
different  operators,  or  on  different  days, 
or  using  different  sets  of  equipment,  then 
a  family  of  closely  related  curves  will  be 
obtained   (Figure  4). 


Figure  4.     Varying  Conditions  Within  a 
Laboratory 


If  similar  series  of  tests  are  made  in 
a  second  laboratory,  a  second  family  of 
curves  relating  measurements  to  quantity  will 
be  obtained  (Figure  5). 


M 


Figure 


Varying  Conditions  Within  and 
Between  Laboratories 


An  independent  measure  of  Q  may  be 
available  in  some  cases,  from  the  manufac- 
turer of  the  samples  (for  example,  concen- 
tration) or  from  results  obtained  by  a 
superior  reference  method.     An  alternative 
approach  -  and  in  most  cases  the  only 
available  approach  -  is  to  represent  the 
quantity  being  measured  by  the  average  value 
obtained  by  a  number  of  laboratories  all 
applying  the  specified  measuring  process. 
B$y  so  doing  the  curves  representing  each 
laboratory  usually  are  linearized  (Figure  6) . 


M 

(one  lab) 


M  (all) 

(averages  all  labs) 


Figure  6.     Linearization  of  Curves  When 
Plotting  One  Laboratory  Against  Averages 
of  All  Laboratories 


If  a  laboratory's  results  agree  exactly 
with  averages  over  all  laboratories,  the 
laboratory  would  be  represented  by  a  45°  line 
through  the  origin  (Figure  7A) .     Generally  a 
laboratory  can  have  a  zero  error  (e.g.,  for 
a  chemical  method,  failure  to  make  a  blank 
correction)   (Figure  7B) ,  an  amplification 
error   (Figure  7C),  or  both  (Figure  7D) . 


M 


M  (all) 


Figure  7.     Possible  Situations:     a)  Labora- 
tory Agrees  with  Averages  Over  All  Labora- 
tories; b)  Laboratory  Has  Zero  or  Constant 
Error;  c)  Laboratory  Has  Amplification  or 
Proportional  Error;  and  d)  Laboratory  has 
Both  Errors 

This  representation  of  the  results  of 
an  inter laboratory  testing  program  (or 
"round  robin"  as  it  sometimes  is  called)  is 
known  as  the  linear  model  [1,  2,   3]1.  The 
deviation  of  the  line  from  45°  through  the 
origin  is  a  systematic  error  consisting  of 
two  components:     the  zero  error  and  the 
amplification  error. 

When  the  procedure  followed  by  a  lab- 
oratory differs  appreciably  from  the  stan- 
dard procedure,   some  non-linearity  may  be 
expected.     Also,   even  if  approximately 
linear,  serious  laboratory-material  inter- 
action may  occur.     What  is  meant  by  labora- 
tory-material interaction  in  the  linear 
model? 


Figures  in  square  brackets  indicate 
the  literature  references  at  the  end  of  the 
paper . 
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When  the  actual  values  obtained  by  a 
laboratory  for  several  materials  are  plotted 
against  the  average  values  obtained  by  many 
laboratories,  the  plotted  points  for  the 
laboratory  usually  do  not  fall  exactly  on  the 
best  fitting  line  for  the  laboratory 
(Figure  8) . 


M 


M  (all) 


Figure  8.     Deviations  of  Individual  Test 
Results  from  Laboratory's  Line 


There  are  three  reasons  why  the  points 
deviate  from  the  line: 

(1)  Sample  variability; 

(2)  Moment-to-moment  instrument/ 
environment/operator  variability; 

(3)  Laboratory-material  interaction. 

The  first  two  of  these  may  be  considered 
random  effects  and  theoretically  can  be 
reduced  as  small  as  desired  by  replication 
(i.e.,  by  replicating  the  measurements  on  a 
series  of  test  pieces  and  averaging  the  test 
observations).     However,  when  this  is  done, 
the  points  often  still  do  not  fall  on  the 
line  because  of  laboratory-material 
interaction. 

It  is  easier  to  give  an  example  of  this 
interaction  than  to  define  it  precisely. 
Consider  two  materials  that  would  give 
exactly  the  same  test  value  if  a  certain 
interpretation  of  the  test  protocol  is 
followed  exactly.     Also,  suppose  that  the 


first  of  these  materials  is  quite  sensitive 
to  relative  humidity  but  the  second  is  not. 
If  a  laboratory  fails  to  control  relative 
humidity  sufficiently  close  to  a  required 
level  (perhaps  because  of  its  interpretation 
of  the  protocol) ,  it  could  obtain  the  correc 
test  value  (or  at  least  a  value  close  to  its 
line)  for  the  second  material  but  not  for 
the  first.     It  can  be  said  then,  that  there 
is  an  interaction  between  the  sensitivity  of 
the  material  to  relative  humidity  and  the 
failure  of  the  laboratory  to  control  relativ 
humidity  close  enough.     The  distance  of  the 
plotted  point  from  the  line  for  this  labora-1 
tory  and  material  will  remain  large  no  matte- 
how  much  replication  is  involved. 


No  matter  how  well  the  test  protocol  is 
standardized  and  how  well  the  laboratories 
follow  the  protocol,  some  laboratory-mater ia 
interaction  will  be  present,  just  as  there 
will  almost  always  be  some  systematic  lab- 
oratory error  and  some  random  replication 
error.     Standardization  does  not  result  in 
perfection,  hopefully  it  does  result  in  the 
reduction  of  these  errors  to  the  level  where  ■• 
they  have  no  practical  significance. 


Evaluation  of  Laboratories 
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in  view  of  the  theory  of  the  measuring 
process,  what  is  it  that  should  be  examined 
in  the  evaluation  of  laboratories?  Ideally 
the  examination  of  a  laboratory  with  regard 
to  its  capability  to  perform  a  specific  test  f5' 
method  should  provide  information  on  its 
systematic  (zero  and  amplification)  errors,  3 
its  ability  to  replicate  its  own  measurement 
and  how  that  varies  with  test  level,  its 
residual  or  interaction  errors  for  each 
class  of  materials,  and  the  likelihood  that  1 
the  laboratory's  quality  of  performance  will 
be  maintained.     The  last  is  a  question  of 
being  in  statistical  control. 


■ith 
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For  the  purpose  of  laboratory  accredita 
tion,  the  capability  of  a  laboratory  to  per- 
form may  be  examined  by  three  methods: 

i 

(1)  Information  gathering,  such  as  a 
questionnaire  completed  by  the 
laboratory  and  evaluated  by 
experts ; 

(2)  On-site  inspection  of  the 
laboratory; 

(3)  Proficiency  testing. 

j, 

As  used  here  a  proficiency  test  is  any 
means  by  which  the  actual  testing  perfor-  , 
mance  of  a  laboratory  is  determined. 
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The  capability  to  perform  might  be 
examined  exclusively  by  proficiency  testing 
in  the  form  of  a  periodic  interlaboratory 
l(or  round  robin)  testing  program  in  which  a 
number  of  materials  are  distributed  simul- 
taneously to  each  of  the  laboratories 
(desiring  accreditation  or  continued  accredi- 
tation.    If  a  sufficient  number  of  materials 
and  replicate  test  specimens  of  each  material 
,are  distributed  to  each  laboratory  and  this 
Is  repeated,  say,  every  few  weeks,  a  very 
complete  picture  of  the  laboratory's  per- 
formance would  be  obtained,  including 
information  on  systematic,  random  and  inter- 
action errors  and  the  stability  of  the 
laboratory's  performance  over  a  period  of 
time.     Unfortunately,  such  a  proficiency 
testing  program  is  likely  to  be  very 
expensive  both  to  administer  and  in  testing 
.time  for  each  laboratory. 

At  the  other  extreme,  in  order  to  assure 
pontrol  over  the  four  elements  that  enter 
into  each  realization  of  the  measuring 
process,  all  laboratories  that  wish  to  be 
Accredited  could  be  required  to  have  exactly 
'the  same  equipment,  calibrated  frequently 
py  the  same  calibration  service.     Also,  all 
personnel  (testing  technicians  and  supervi- 
sors) could  be  required  to  have  the  same 
training,  perhaps  in  a  special  course 
established  by  the  accrediting  authority, 
iven  then,  it  might  be  necessary  to  be  con- 
cerned with  environmental  factors  such  as 
gravity  and  barometic  pressure.     Can  a 
laboratory  located  in  Denver  obtain  the  same 
test  results  as  one  located  close  to  sea 
ILevel?     Obviously  if  all  laboratories  are 
ising  identical  equipment  and  identically 
trained  personnel,  it  is  more  likely  that 
l:hey  will  obtain  equivalent  test  results, 
Ifith  relatively  small  systematic  and  inter- 
action errors.     Again,  unfortunately,  such 
requirements  are  likely  to  be  expensive  and 
aoreover  tend  to  freeze  the  state-of-the-art. 

Between  these  extremes  are  various 
possible  combinations  and  types  of  require- 
ments, questionnaires,   inspections  and  pro- 
ficiency testing,   the  best  combination  vary- 
ing with  the  nature  of  the  test  method,  the 
lumber  of  laboratories  and  so  forth.  For 
example,  with  heavy  dependence  on  an  exten- 
sive questionnaire  to  determine  details 
concerning  a  laboratory's  personnel, 
equipment,   quality  control  procedures, 
idequacy  of  financing,  and  so  forth,  and  on 
a  thorough  inspection  by  trained  inspectors, 
a  single  proficiency  test  sample  might  be 
ised.     From  the  information  obtained  about 
test  equipment  and  procedures  by  the 
juestionnair e  and  inspection,   experts  should 
)e  able  to  judge  the  likely  performance  of 


the  laboratory.     The  test  result  obtained  by 
the  laboratory  in  testing  the  proficiency 
sample  will  either  verify  this  judgment  or 
it  will  not.     If  not,  the  result  cannot  dis- 
tinguish between  a  systematic  deviation,  a 
random  error,  an  interaction,  or  a  poor 
sample.     A  disqualified  laboratory  will 
certainly  claim  the  last. 

One  combination  that  should  work  well 
in  many  situations  includes  for  each  test 
method : 

(1)  A  small  number  of  pertinent 
questions  about  the  equipment  and 
its  calibration,  with  the  response 
of  the  laboratory  evaluated  by 
experts.     The  experts  may  generate 
additional  questions  either  for 
mail  response  or  for  the  inspec- 
tion. 

(2)  A  quick  inspection  to  verify  the 
existence  of  the  claimed  equip- 
ment and  calibration  records,  the 
general  condition  of  the  equip- 
ment, and  one  or  two  critical 
dimensions,   the  last  being 
especially  important  if  equipment 
design  requirements  have  been 
changed  over  the  years. 

(3)  A  periodic  proficiency  test,  the 
size  and  frequency  of  which  de- 
pending on  the  cost  of  testing 
and  samples,  and  the  likely 
stability  of  the  equipment.  The 
proficiency  test  should  include 
at  least  two  test  samples,  one 
each  of  two  very  similar  but  not 
identical  materials.     Such  a 
minimal  proficiency  test  provides 
for  each  laboratory  just  one 
degree  of  freedom  for  calculation 
of  systematic  plus  interaction 
error  and,  if  the  materials  are 
sufficiently  similar,  approximately 
one  degree  of  freedom  for  replica- 
tion error.     This  minimum  program 
cannot  provide  information  on  how 
the  systematic  error  varies  with 
test  level  nor  how  much  inter- 
action error  is  present. 

The  above  two-sample  proficiency  test 
was  recommended  by  Dr.  Jack  Youden  [4,  5] 
and  is  used  in  some  of  our  Collaborative 
Reference  Programs  which  were  established  to 
help  a  participating  laboratory  evaluate  its 
performance  in  comparison  with  that  of  other 
laboratories.     The  Youden  approach  has  been 
very  successful  because  of  its  easily  under- 
stood graphical  presentation  (Figure  9)  in 
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which  each  laboratory  is  represented  by  a 
point  on  the  graph,  the  point  being  at  the 
intersection  of  the  coordinates  of  the  lab- 
oratory's results  for  the  two  samples.  As 
the  points  for  95%  of  similar  laboratories 
should  fall  within  the  ellipse,  the  perfor- 
mance of  a  laboratory  with  a  point  appreciably 
outside  the  ellipse  is  very  questionable. 


[3]     Lashof,  T.  W. ,  Precision  of  Methods  foi  : 
Measuring  Tensile  Strength,  Stretch, 
and  Tensile  Energy  Absorption  of  Paper 
Tappi,  Vol.  46,  No.  1,  January  1963, 
pp.  52-59. 

[4]     Youden,  W.  J.,  Graphical  Diagnosis  of 
Inter  laboratory  Test  Results,  Industri; 
Quality  Control,  Vol.  15,  No.  11,  May 
1959,  1-5. 

[5]     Mandel,  J.  and  Lashof,  T.  W. ,  Inter- 
pretation and  Generalization  of  Youden 
Two  Sample  Diagram,  Journal  of  Quality 


Technology,  Vol.  6,  No.  1,  January  197 
pp.  22-36. 
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Figure  9.     Youden  Two-Sample  Diagram 


What  has  been  presented  here  is  quite  sim- 
plified compared  with  the  real  world  of 
laboratory  evaluation,  but  an  understanding 
of  the  theory  of  the  measuring  process  as 
presented  here,  or  a  suitable  extension  of 
it  for  a  specific  testing  situation,  is  very 
helpful  in  the  design  of  appropriate 
examination  procedures  for  the  evaluation  of 
laboratories . 
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INTER- LABORATORY  ROUND  ROBINS  FOR  DETERMINATION  OF 
ROUTINE  PRECISION  OF  METHODS 

CHARLES  A.  BICKING 

TRACOR- JITCO ,  INC. 
1776  E.  JEFFERSON  ST. 
ROCKVTLLE,  MARYLAND  20852 

Attention  is  directed  to  the  importance  of  determining  precision 
of  test  methods  under  routing  conditions  of  use.     Comparisons  of  pre- 
cision values  obtained  under  very  limited  conditions  have  a  very  nar- 
row range  of  usefulness.     Such  "ideal"  precision  values  lead  to  incor- 
rect use  of  "checking  limits"  in  attempts  to  control  performance  of 
operators  and  of  laboratories  and  of  statistically  invalid  rules  for 
retesting,  rejection  of  results,  and  initiation  of  third-party  referee 
testing.    Measures  of  precision  under  routine  conditions,  both  within- 
laboratory  and  between-laboratories ,  are  necessary  to  identify  the 
contribution  of  the  test  method  to  the  total  variability  of  measurement 
in  research,  manufacturing,  and  in  exchange  of  materials  and  goods  in 
commerce . 

Key  words:     Accuracy;  between-laboratory  precision;  components  of 
variance;  "duplicity";  ideal  conditions;  measurement  process;  "omni- 
fariousness" ;  precision;  round  robin;  routine  conditions;  variance 
model;  within-laboratory  precision. 


1.  Introduction 

An  essential  preliminary  to  the  plan- 
ning of  an  inter-laboratory  round  robin  is 
agreement  on  the  uses  to  be  made  of  the  re- 
sulting precision  information.     This  is  so 
jbecause  there  are  many  suitable  models  on 
which  design  of  the  round  robin  may  be  based. 
Proper  use  of  the  information  depends  on  the 
components  that  have  been  included  in  the 
model.     If,  for  example,  within-laboratory 
precision  is  determined  by  one  operator, 
using  one  instrument,  making  repeated  tests 
on  a  standard  sample  within  a  short  period 
of  time,  ideal  conditions  may  have  been  rep- 
resented, but  the  result  will  be  of  little 
use  in  describing  agreement  of  test  results 
when  the  test  method  is  used  in  research, 
manufacturing,  or  in  the  exchange  of  ma- 
terials in  commerce.     Precision  obtained 
under  ideal  conditions  has  limited,  tightly 
circumscribed  uses  in  the  development  of 
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test  methods,  comparison  of  competing  meth- 
ods, and  in  evaluating  modifications  to 
methods.     However,  the  best  possible  agree- 
ment of  results  under  ideal  conditions  has 
little  relevance  to  agreement  among  measure- 
ments in  routine  use  of  test  methods. 

2.    Measurement  as  a  Process 

The  measurement  of  a  property  of  a  ma- 
terial should  be  looked  upon  as  a  process. 
The  major  elements  in  the  process  are  ma- 
terials, methods,  instruments  (or  reagents), 
operators,  time,  environmental  conditions, 
and  laboratories. 

A  variance  model  is  applied  to  the  pro- 
cess.   The  total  variance  is  made  up  of  com- 
ponents due  to  the  elements  which  make  up 
the  process.     The  process  may  be  truncated 
at  any  point.     The  model  may  be  as  simple 
as  the  one  representing  the  ideal  conditions 


mentioned  in  the  Introduction;  a  standard 
sample,  tested  two  or  more  times  by  the  same 
operator,  using  the  same  instrument  over  a 
very  short  period  of  time.     The  resulting 
precision  information  is  useful  only  for 
describing  this  particular,  truncated  mea- 
surement process.     It  is  not  applicable  for 
any  comparison  with  measurements  made  by 
other  operators  using  other  instruments  at 
different  times. 

On  the  other  hand,  the  model  may  in- 
volve all  the  recognized  elements  or  vari- 
ables; two  or  more  samples  of  a  material, 
each  tested  more  than  once  by  each  of  two  or 
more  operators  using  different  instruments 
(of  the  same  or  different  design),  at  dif- 
ferent times,  under  different  environmental 
conditions  and  in  different  laboratories. 
The  model  may  call  for  omitting  any  compo- 
nent, or  can  combine  components  in  any  mix. 
It  is  extremely  important  to  know  exactly 
what  model  is  used  in  a  round  robin  program. 

The  way  components  are  combined  deter- 
mines how  precision  information  may  be  used 
to  test  the  reality  of  observed  differences. 
In  practical  applications  of  a  test  method, 
we  are  interested  in  the  total  contribution 
of  the  measurement  process  to  the  apparent 
variations  in  observations  made  on  the  pro- 
duct.    We  know  that  in  use  of  the  method 
operators  vary,  instruments  vary,  that  ma- 
terials are  not  perfectly  homogeneous  and, 
perhaps  most  obviously  of  all,  that  it  is 
difficult  to  repeat  results  time  after  time. 
The  extent  to  which  each  of  these  elements 
does  vary  in  the  measurement  process  must 
be  taken  into  account. 

3.     Control  of  the  Elements  of  the 
Process 

Control  is  partly  in  the  hands  of  the 
planners  of  the  round  robin,  partly  within 
each  of  the  participating  laboratories,  and 
partly  in  the  purview  of  the  laboratories  as 
a  group. 

In  the  first  instance,  the  planning 
group  needs  assurance  that  the  material 
distributed  for  test  is  as  homogeneous  as 
possible.     Evidence  of  this  homogeniety  may 
be  provided  by  assurance  that  the  production 
process  was  in  statistical  control.  Even 
with  that  assurance,  samples  for  test  should 
be  distributed  at  random. 

3.1    Operator  Control 

Operator  control  is  maintained  through 
plotting  a  control  chart  for  each  operator 
in  the  laboratory.     It  is  sometimes  suggested 
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that  this  control  chart  be  based  on  dupli- 
cate tests  run  periodically  on  routine  sam- 
ples.    A  control  chart  for  the  range  of 
duplicates  is  kept.     The  operator  is  con- 
sidered to  be  in  control  when  all  ranges  ar 
within  the  control  limits.     This  gives  one 
measure  of  operator  performance,  that  is, 
how  well  results  can  be  duplicated,  on  the 
average,  when  measurements  are  made  within 
a  very  short  period  of  time.     One  danger  is 
that  the  results  may  not  be  independent.  A 
more  serious  deficiency  in  this  approach  is.: 
that,  as  experience  invariably  demonstrates 
the  real  problems  that  operators  have  are 
with  changes  that  occur  in  performance  of 
the  test  over  time.     These  changes  are  due  1 
to  differences  in  set-up  and  calibration  of 
instruments,  changes  in  reagents,  tempera- 
ture or  humidity  changes ,  fluctuations  in 
laboratory  services  (water,  electricity, 
etc.)  or  other  uncontrolled  or  unrecognized 
variables . 

Therefore,  it  is  recommended  that  oper 
ator  control  be  based  on  control  charts  for 
averages,  as  well  as  ranges,  of  periodic 
replicate  tests  on  portion  of  a  standard 
sample.     Incidentally,  a  standard  or  a 
spiked  sample  is  also  useful  in  measuring 
the  accuracy  of  a  method.     Whereas  precisio 
refers  to  mutual  agreement  among  repeated 
measurements,  accuracy  refers  to  the  agree- 
ment of  the  average  of  observations  with  a 
standard  or  accepted  value.     The  standard 
deviation  provides  an  inverse  measure  on 
precision;  the  constant  error,  or  bias,  is 
an  approximation  of  the  accuracy. 

3.2  Instrument  Control 

Instrument  control  is  achieved  by  reg- 
ular recalibration  of  instruments  and  throu 
development  and  use  of  standard  curves.  Al 
so,  it  is  possible  to  follow  the  success  of 
these  activities  by  maintaining  a  control 
chart  on  the  instrument,  handled  in  a  way 
similar  to  the  operator  control  chart. 

3.3  Time  Control 

The  effects  of  time  may  be  evident  in 
uneven  performance  by  the  operator  or  by  an 
instrument  but  must  also  be  watched  as  a 
phenomenon  affecting  everything  that  goes 
on  in  the  laboratory.     Points  out  of  contro: 
on  the  operator  control  chart  for  averages 
may  be  due  to  an  operator  fault  or  to  some  j 
environmental  change.     The  grouping  of  test 
replicates  and  their  spacing  in  time  should 
be  based  on  a  rationalization  of  possible 
occurrences  in  the  laboratory  environment. 
The  control  plan  must  be  developed  with  ful 
attention  to  the  theory  of  statistical 
control. 


Experience  has  shown  that  in  a  well-run 
laboratory,  good  operator  and  instrument  con- 

-  trol  are  achievable.     Time  control  may  be 
more  difficult.     It  is  not  possible  to  tell 

| exactly  how  much  variability  is  due  to  oper- 
X ator  and  instrument  unless  these  components 
| are  included  in  the  round  robin  model.  It 
I  is  often  assumed  that  they  are  controlled 
i  and  that  their  contribution  to  the  total 
J  variance  is  small.     The  assumption  is  made 
§  that  this  small  variance  is  due  to  system 
|  causes  that  are  built  into  the  test  method 
I  at  the  time  it  was  developed.     Smaller  oper- 
3  ator  or  instrument  variability  (after  all 
I  points  out  of  control  have  been  assigned  a 

cause  and  the  causes  removed)  would  require 

further  developmental  work  on  the  method. 
I  Round  robin  models  intended  to  produce  in- 
|j  formation  useful  in  routine  application  of 

the  method  should  usually  include  performance 
[ of  the  method  on  more  than  one  day,  i.e. , 
:  time  should  always  be  one  component  in  the 

model . 

i  3.^    Inter-laboratory  Control 

I 

Inter-laboratory  control  is  attained  by 
ij  use  of  a  standard  sample  testing  program  or 
the  exchange  of  samples  in  a  proficiency 
! testing  program.     Such  programs  are  widely 
a  used  among  laboratories  belonging  to  one  or- 
i  ganization  or  an  association  of  organiza- 

-  tions.     Unfortunately,  they  do  not  exist 
widely  among  independent  laboratories  join- 
ing together  in  a  round  robin  with  the  ex- 
plicit purpose  of  measuring  precision  of 

'!  test  methods . 

To  an  extent,  use  of  a  pre-test  before 
running  the  main  round  robin  will  serve  the 
purpose  of  control.     However,  control  is  an 
•  ongoing  activity  and  short-term  attempts  to 
j  locate  and  correct  causes  of  large  inter- 

-  laboratory  differences  are  only  partially, 
or  even  only  marginally  effective.  There 
are  no  short-cuts  to  good  control. 

h.     Avoiding  the  Pitfalls 

The  shortcoming  of  the  standard  mater- 
j  ial,  one  operator/instrument,  short  period 
■  of  time  model  is  that  it  results  in  what 
i  W.  J.   (Jack)  Youden  dubbed  "chemist's  du- 
plicity".    Such  a  model  usually  results  in 
a  more  optimistic  level  of  precision  than 
'  when  the  model  calls  for  independent  ob- 
.  servations  performed  on  different  days.  The 

within-laboratory  part  of  the  "duplicity" 
;  model  is  usually  accompanied  by  a  between- 
■I  laboratory  part.     The  between-laboratory 
part  then  confounds  all  differences  between 
operators,  instruments,  and  time  with  real 

-  laboratory  differences.     This  is  an 
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"omnifarious"  model  which  leads  to  an  impos- 
sible situation  when  it  comes  to  analyzing 
the  reasons  for  differences  between  labora- 
tories. 

The  "duplicity  model"  may  be  useful  in 
comparing  one  method  with  another  or  mea- 
suring the  effect  of  a  modification  of  a 
method.     Care  must  be  taken  that  exactly  the 
same  model  is  applied  to  both  trials.  Also, 
this  model  may  be  useful  in  maintaining  the 
precision  of  a  method  (determining  whether 
it  has  changed),  but,  again,  the  model  must 
be  strictly  followed  each  time  it  is  checked 

The  problem  with  publishing  the  ideal 
precision  in  a  method  is  that  there  is  an 
implication  that  any  single  pair  of  results 
run  in  any  lab  at  any  time  should  have  as 
good  a  result.     It  becomes  particularly  rep- 
rehensible when  precision  information  of  thi 
sort,  whether  expressed  as  a  standard  devia- 
tion or  as  a  "checking  interval"  is  used  to 
try  to  control  an  operator  or  a  laboratory. 
See  Section  3  on  control  of  the  elements  of 
a  measurement  process.     At  best,  these  num- 
bers can  be  used  only  as  a  very  rough  guide 
as  to  whether  a  new  operator  or  a  new  labor- 
atory is  in  the  general  region  of  precision 
achieved  in  good  laboratories.     Even  as  a 
rough  guide,  the  limitations  of  this  model 
must  be  recognized. 

Unfortunately,  when  "checking  inter- 
vals" are  presented  in  a  method,  users  of 
the  method  are  encouraged,  in  a  completely 
indefensible  way,  to  use  the  intervals  to 
call  for  retesting,  for  accepting  the  re- 
test  with  or  without  regard  for  the  original 
result,  or  for  justifying  comparative  test- 
ing programs  or  the  employment  of  a  referee 
laboratory.     None  of  these  things  are  indi- 
cated on  the  basis  of  such  extenuated  evi- 
dence . 

With  so  little  to  recommend  the  prac- 
tice of  using  minimum  models,  it  is  hard  to 
explain  why  more  attention  has  not  been 
given  to  precision  in  the  routine  use  of 
methods . 

5.     A  Useful  Routine  Precision  Model 

Prerequisites  for  a  precision  model  for 
use  in  round  robin  testing  include  existence 
of  a  test  method  which  has  gone  through  a 
shakedown  period  in  one  laboratory,  rugged- 
ness  testing  to  demonstrate  that  it  is  not 
sensitive  to  minor  perturbations  in  test 
conditions,  and  which  exhibits  control  with- 
in laboratories.     It  is  also  desirable  that 
it  have  been  incorporated  in  an  inter-lab- 
oratory proficiency  testing  program. 


Because  statistical  control  of  the  meth- 
od has  not  always  been  recognized  as  a  re- 
quirement in  all  laboratories,  it  is  desir- 
able that  a  preliminary  round  or  rounds  be 
run  to  familiarize  all  participants  with  the 
method  and  to  work  out  any  bugs  which  still 
exist . 

Assuming  that  attention  has  been  given 
to  such  preliminaries,  a  number  of  labora- 
tories, not  less  than  six  and  preferably  at 
least  10  should  have  agreed  to  praticipate 
in  the  round  robin.     Homogeneous  samples  of 
material  must  have  been  obtained  and  distrib- 
uted at  random  among  the  participating  labor- 
atories . 

If  the  method  is  to  be  used  to  test  more 
than  one  material,  a  number  of  different  ma- 
terials should  be  sampled  so  that  the  whole 
range  of  use  of  the  test  will  be  represented. 
Structural  differences  in  materials,  inter- 
ferences of  possible  contaminants,  and  other 
things  that  could  account  for  a  variable  re- 
sponse should  be  considered  in  selecting  the 
materials.     If  the  test  applies  to  only  one 
material,  samples  should  be  supplied  at  dif- 
ferent levels  of  content  of  the  property  to 
be  measured,  for  example,  at  different  con- 
centrations.    Precision  may  be  constant  at 
all  levels,  may  be  proportional  to  the  level, 
or  may  respond  non-linear ly  (logarithmicly, 
exponentially,  etc.)    At  least  three  levels, 
and  preferably  more,  are  necessary  to  mea- 
sure curvature. 

Assuming  that  operators  and  instruments 
are  controlled  and  their  contribution  to  the 
total  variation  in  the  measurement  process 
is  small,  a  useful  model  is  to  have  one  op- 
erator in  each  laboratory  make  duplicate 
determinations  on  each  sample  on  each  of  two 
days.    The  within-laboratory  precision  will 
be  made  up  of  components  due  to  the  dupli- 
cates and  due  to  days.    The  between-labora- 
tory precision  will  be  made  up  of  the  within- 
laboratory  precision  plus  the  component  due 
to  laboratories. 

At  least  the  within-laboratory  and  be- 
tween-laboratory standard  deviations  should 
be  presented.     If  for  either  of  the  categor- 
ies of  precision,  the  values  are  the  same 
(not  significantly  different)  for  all  mater- 
ials or  for  all  levels  of  a  material,  only 
a  single  value  need  be  reported  for  within- 
and  one  for  between-laboratory  precision. 
If  the  standard  deviations  are  proportional 
to  the  level,  the  precision  may  be  reported 
as  the  coefficient  of  variation;  the  standard 
deviation  divided  by  the  average  times  100. 
If  the  precision  varies  with  material  or 
level,  a  table  may  be  prepared  grouping 


materials  or  levels  with  the  same  (not  sig- 
nificantly different)  precision.     If  the 
precision  is  related  to  the  level  curvilin- 
early,  the  results  may  be  presented  as  a 
curve . 

6.  Conclusion 

The  degree  of  skill  employed  in  plan- 
ning a  round  robin  is  indicated  by  how  well 
the  planner  has  succeeded  in  avoiding  "du- 
plicity" and  "omnif ar-ousness" .     The  model 
should  be  directly  aimed  at  the  purpose  to 
which  the  precision  information  is  to  be 
put,  how  and  in  what  environment  the  test 
method  is  to  be  used.     Since  standard  meth- 
ods of  test  are  intended  for  use  in  all  kirn 
of  practical  situations,  it  is  imperative 
that  inter-laboratory  round  robins  be  planm 
for  determination  of  precision  of  methods  ui 
der  routing,  not  ideal  conditions. 
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Recognizing  the  need  to  ensure  the  validity  of  environmental  data, 
Monsanto  has  charged  its  Environmental  Analytical  Sciences  Center  (EASC) 
with  ensuring  that  all  analyses  for  monitoring  the  environmental  effects 
of  its  operations  are  credible.     Thus  far,  the  effort  has  focused  on  the 
credibility  of  the  industrial  hygiene  analyses  performed  throughout  the 
company.     Since  round-robin  testing  programs  are  an  effective  tool  to 
demonstrate  the  validity  of  analytical  measurements,  the  EASC  established 
a  Monsarto  Analytical  Testing  Program.     Under  this  program,  modeled  after 
NIOSH's  PAT  Program,  the  EASC  periodically  supplies  standard  samples  of 
compounds  of  interest  to  various  company  laboratories.     Standard  samples 
of  materials,  unavailable  elsewhere,  are  prepared  on  sampling  substrates 
commonly  employed  by  Monsanto  laboratories. 

Two  compounds  of  major  interest  to  Monsanto  are  acrylonitrile  (AN) 
and  styrene  monomer  (SM) .     Two  methods  are  employed  for  workplace  monitor- 
ing of  these  compounds.     Some  sites  employ  a  modification  of  the  standard 
NIOSH  method  using  charcoal  tubes  as  the  sampling  medium.     In  this  method 
the  sample  is  desorbed  with  a  solvent  prior  to  gas  chromatographic  deter- 
mination.    The  remaining  sites  employ  sample  collection  tubes  packed  with 
either  Porapak  N  or  Chromosorb  101.     In  this  method  the  sample  is  thermally 
desorbed  from  the  porous  polymer  tubes  and  injected  directly  into  a  gas 
chromatograph.     Generating  standard  samples  of  AN  and  SM  on  these  two  sub- 
strates requires  different  approaches.     The  charcoal  tube  standards  are 
generated  on  a  high-flow  manifold  at  collection  rates  of  between  100-1000 
cc/minute  using  a  high-rate  permeation  source.     Porous  polymer  tube  samples 
are  generated  on  a  low-flow  collection  manifold  (10-20  cc/min.)  using  low- 
rate  permeation  tube  devices. 

This  paper  describes  the  Monsanto  Analytical  Testing  Program,  the 
standard  generation  system  employed,  and  the  problems  encountered  in  gen- 
erating known  standards.     The  steps  involved  in  documenting  the  composition 
of  the  standard  samples  and  the  results  of  several  round-robin  evaluations 
will  also  be  presented. 

Key  words:     Acrylonitrile;  industrial  hygiene;  proficiency  testing;  round- 
robin  standards  generation. 
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Good  laboratory  practices  regulations 
promulgated  by  the  FDA  [l]1  and  proposed  by 
the  EPA  [2]  will  affect  the  way  health 
effects  testing  laboratories  conduct  anal- 
yses.    The  EPA  and  OSHA  have  also  ex- 
pressed concern  about  the  quality  of  data 
being  submitted  by  environmental  and 
industrial  hygiene  testing  laboratories. 
GLP  regulation,  laboratory  certification 
and  accreditation  schemes  have  come  into 
being  as  a  result  of  concerns  of  these 
governmental  agencies. 

Because  most  of  Monsanto' s  plant 
laboratories  submit  industrial  hygiene  and 
environmental  data  to  state  and  Federal 
agencies,  it  is  important  that  these  data 
comply  with  GLP  concepts  and  be  based  on 
sound  scientific  procedures.     To  assure 
that  its  plant  laboratories  produce  data 
of  documented  validity,  Monsanto  began  an 
aggressive  good  laboratory  practice  pro- 
gram by  establishing  a  good  laboratory 
practices  group  in  the  Environmental 
Analytical  Sciences  Center  (EASC)  at 
Dayton  in  January  1978. 

The  programs  established  by  this 
group  includes: 

1.  Preparing  a  manual  spelling  out 
the  essential  elements  of  Good 
Laboratory  Practices. 

2.  Encouraging  each  plant  laboratory 
to  prepare  a  GLP  manual  which 
describes  their  site  program. 

3.  Encouraging  a  voluntary  GLP 
laboratory  audit  program. 

4.  Field  validation  and  audit  of  auto- 
matic continuous  area  monitors  used 
to  analyze  plant  environments. 

5.  Helping  plant  laboratories  work 
towards  American  Industrial  Hygiene 
Association  accreditation.  The 
ultimate  goal  is  to  have  one  plant 
lab  accredited  in  each  of  the  five 
Monsanto  operating  companies  (two 
have  AIHA  accreditation  at  this 
time) . 


■'■Figures  in  brackets  indicate  the 
literature  references  at  the  end  of  the 
paper. 


6.  Setting  up  a  standards  preparation 
laboratory  which  tailors  standards  to 
specific  needs  of  Monsanto  plants. 

7.  Interlaboratory  proficiency  testing  of 
plant  labs  by  holding  round-robin  test? 
of  compounds  of  interest  to  Monsanto.  I 
This  program  has  been  dubbed  the 
Monsanto  Analytical  Testing  (MAT)  pro-j 
gram,  and  is  modeled  after  the  National 
Institute  of  Occupational  Safety  and 
Health's  Proficiency  in  Analytical 
Testing  (PAT)  program. 

1.     The  MAT  Program 

One  of  the  minor  elements  of  a  com- 
prehensive GLP  program  is  laboratory  parti- 
cipation in  round-robin  studies.  However, 
the  majority  of  round-robin  studies  avail- 
able are  not  applicable  to  Monsanto  Company 
laboratories.     That  is,  the  types  of  contam 
inants,  samples  matrices  and  sampling  sub- 
strates available  are  not  relevant  to  the 
majority  of  analyses  encountered  by  Monsant 
environmental  laboratories.     To  fill  this 
special  need,  the  EASC  established  a  stan- 
dards generation  facility.     The  mission  of 
the  facility  is  to  prepare  environmental 
standards  that  are  pertinent  to  the  analyse? 
commonly  performed  within  the  company. 

Participation  in  the  program  is  com- 
pletely voluntary.     Results  are  reported  in 
a  coded  format  to  ensure  anonymity.  All 
supporting  data  relating  to  the  homogeneity 
and  stability  of  the  standard  samples  is 
reported  to  participating  laboratories  with 
the  results  of  the  study.     The  philosophy  of 
result  reporting  is  one  of  supplying  the 
laboratories  with  sufficient  data  to  enable 
them  to  make  a  judgement  about  their  labora 
tory's  performance  rather  than  grading  or 
ranking  laboratories. 

To  date  the  effort  has  focused  on  the 
credibility  of  the  industrial  hygiene 
analyses  performed  throughout  the  company. 
Two  compounds  of  major  interest  to  Monsanto 
are  acrylonitrile  (AN)  and  styrene  (SM) . 
Both  of  these  compounds  are  monitored 
intensively  at  various  plants  throughout  th< 
company.     Some  laboratories  may  analyze  up 
to  100  samples  per  month.     It  is  thus  impor- 
tant to  demonstrate  the  validity  of  these 
analyses,  not  only  from  the  viewpoint  of 
worker  safety,  but  also  to  provide  a  measurt 
of  data  validity  since  decisions  concerning 
plant  operating  procedures  are  sometimes 
based  upon  industrial  hygiene  measurements. 
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Within  Monsanto,  two  distinct  methods 
are  employed  for  workplace  monitoring  of  AN. 
Some  sites  employ  a  modification  of  the 
standard  NIOSH  method  using  charcoal  tubes  as 
the  sampling  medium.     Other  sites  employ 
sample  collection  tubes  packed  with  either 
Porapak  N  or  Chromosorb  101.     The  charcoal 
tube  method  samples  a  larger  volume  (=20  L) 
of  air  and  employs  solvent  desorption 
(usually  2%  acetone  in  CS2) .     The  porous 
polymer  tube  method  samples  a  much  smaller 
volume  of  air  (^3  L)  and  employs  thermal 
desorption  of  the  sample  directly  into  a 
chromatograph.     Sampling  an  atmosphere  of  2 
ppm  AN,  which  is  the  threshold  limit  value 
(TLV) ,     the  charcoal  and  the  porous  polymer 
tube  methods  collect  about  86  and  13  yg  of 
AN,  respectively. 

2.     Generation  of  Standard  Samples 
Containing  Acrylonitrile 

Standard  samples  are  prepared  by  drawing 
a  known  volume  of  a  dynamically  generated 
atmosphere  of  AN  through  the  sampling  tubes. 
The  AN  atmosphere  is  generated  with  a  Kin-Tek 
Precision  Calibration  System  equipped  with  a 
permeation  source.     Pure  air  is  supplied  with 
an  Aadco  Model  737  pure  air  generator.  The 
AN  contaminated  air  stream  moves  from  the 
permeation  device  and  is  diluted  with  addi- 
tional air  before  it  passes  through  a  glass 
mixing  chamber  to  a  multiport  glass  manifold. 
(See  Figure  1).     The  diluted  air  stream  is 
drawn  through  sampling  tubes  attached  to  the 
glass  manifold  by  a  sampling  plenum.  The 
sampling  plenums  were  constructed  in  a  sim- 
ilar manner  to  those  used  by  the  National 
Bureau  of  Standards  [3]. 


Figure  1.     Schematic  of  Standards 
Generation  System 


In  the  case  of  charcoal  tube  samples, 
the  sampling  plenum  consists  of  three  sets  of 
|  flow  limiting  orifices  (hypodermic  needles) . 
Four  matched  orifices  comprise  each  set.  The 
orifices  within  each  set  were  selected  such 
that  the  individual  flowrates  do  not  vary  by 
more  than  ±2%  from  the  set  averages. 


The  average  flowrates  of  the  three  sets  of 
orifices  are  0.933,  0.440  and  0.110  L/min. 
Using  this  apparatus  (Figure  2) ,  three 
different  loadings  of  AN  on  the  charcoal 
tubes  can  be  obtained  simultaneously.  For 
this  study,  the  desired  amounts  of  AN 
loaded  onto  the  sample  tubes  are  approxi- 
mately 11,  43,  and  90  yg  of  AN;  simulating 
the  amounts  that  would  be  collected  from  20 
L  of  air  containing  approximately  0.2,  1.0 
and  2  ppm  of  AN  respectively. 


Figure  2.     Charcoal  Tube  Sampling 
Plentum  (one  orifice  assembly  shown) 


For  porous  polymer  tube  samples,  the 
sampling  plenum  (Figure  3)  consists  of 
twenty  flow- limiting  orifices  (33  gage 
hypodermic  needles)  carefully  selected  to 
provide  flowrates  of  within  ±1.5%  of  one 
another.     Figure  4  shows  the  construction 
of  an  individual  orifice  assembly.  The 
average  flowrate  of  these  matched  orifices 
is  12.28  mL/min. 
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Figure  3.     Porous  Polymer  Tube  Sampling 
Plentum  (four  orifices  shown) 


A  lower  flowrate  is  required  for 
polymer  tube  samples  since  the  pressure 
drop  across  the  packed  tube  increases 
dramatically  with  increasing  flowrate.  At 
a  flowrate  of  less  than  approximately  35 
mL/min  the  pressure  drop  across  a  tube 
becomes  negligible  in  comparison  to  the 
pressure  drop  across  the  orifice.  A 
constant  flowrate  is  obtained  regardless  of 
the  flow  characteristics  of  the  sample 
tube.     Varying  amounts  of  AN  are  deposited 
on  the  tubes  by  varying  the  collection 
time.     Usually,  the  collection  time  is 
adjusted  to  obtain  either  6  or  13  yg  of 
AN,  simulating  the  amounts  that  would  be 
collected  from  3  L  of  air  containing 
approximately  1.0  or  2.0  ppm  of  AN,  re- 
spectively. 


Figure  4.     Individual  Orifice  Assembly 


Early  in  the  program,  the  sampling 
plenum  was  constructed  with  fine  metering 
valves  in  place  of  hypodermic  needle  or- 
ifices.    This  configuration  was  abandoned 
because  the  metering  valves  were  found  to 
require  periodic  calibration  and  readjust- 
ment in  order  to  maintain  a  set  flowrate 
variability  of  not  greater  than  ±2%. 
However,  this  type  of  apparatus  will  hold 
its  calibration  for  several  days  and  was 
used  to  produce  the  first  set  of  porous 
polymer  tube  samples  used  in  MAT  Round  1. 

3.     Standard  Generation  Protocol 

Before  standard  samples  can  be  em- 
ployed in  a  round-robin  study,  an  assess- 
ment of  sample  integrity  must  be  under- 
taken.    EASC  uses  the  following  general 
protocol  to  ensure  the  production  of 
acceptable  standards: 
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A.  Determine  the  precision 
and  accuracy  of  the  analytical  tech- 
nique employed  for  sample  quality 
control.     This  determination  is  made 
at  each  contaminant  level. 

B.  Determine  the  reproducibility  of  the 
generated  standards  by  producing  and 
analyzing  several  sets  of  standards  at 
the  desired  loading  levels.  If 
possible,  verify  the  accuracy  of  the 
generated  standards  by  using  an 
independent  method(s)  or  standard(s) 
to  determine  the  contaminant  loadings. 
Ideally,  the  independent  method (s)  or 
standard (s)  should  have  clear  trace- 
ability  to  the  National  Bureau  of 
Standards.     If  no  independent  method 
or  standard  is  available,  have  a  set 
of  standards  analyzed  by  a  different 
laboratory  to  verify  loadings. 

C.  Determine  the  stability  of  the  gener- 
ated standard  over  at  least  a  two  week 
period.     Ideally,  the  standard  should 
be  stable  at  room  temperature  for  at 
least  two  weeks.     However,  in  some 
cases,  sample  preservation  steps 
(e.g.,  refrigeration)  may  be  nec- 
essary. 

D.  Based  upon  the  observed  contaminant 
loadings  obtained  in  step  B  and/or  C, 
calculate  the  average  observed  loading 
at  the  99%  confidence  level. 

4.     Results  of  the  Acrylonitrile 
Round-Robin  Studies 

Two  AN  round-robin  studies  have  been 
completed.     Six  laboratories  participated 
in  the  AN  on  Porapak  N  study.     Ten  labora- 
tories were  involved  in  the  AN  on  charcoal 
round-robin.     While  the  majority  of  parti- 
cipants were  Monsanto  Company  Laboratories, 
four  laboratories  outside  of  the  company 
participated  in  the  charcoal  tube  study. 
In  both  studies,  each  laboratory  received 
three  samples  and  a  blank.     Two  of  the 
samples  were  identical,  simulating  samples 
contaminated  to  the  action  level  (one-half 
the  TLV  level).     The  third  sample  simulated 
a  sample  contaminated  to  the  TLV  level. 
The  duplicate  loadings  were  submitted  to 
serve  as  a  rough  check  on  intralaboratory 
precision.     The  results  of  the  two  studies 
are  given  in  Tables  1  and  2.     For  both 
studies,  the  data  indicate  individual 
laboratories  have  a  good  degree  of  intra- 
laboratory precision.     The  average  relative 
deviation  from  the  average  of  the  two 
identical  samples  was  1.5%  for  the  Porapak 
N  round-robin  participants  and  2.2%  for  the 
charcoal  round-robin  participants. 
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The  interlaboratory  agreement  was  not  as 
good.     Laboratories  in  the  Porapak  N  study 
had  relative  standard  deviation  of  19.3%  and 
17%  for  samples  similating  the  action  and  TLV 
levels,  respectively.     Laboratories  partici- 
pating in  the  charcoal  study  displayed  a 
slightly  larger  standard  deviation  of  24.5% 
and  25.4%  for  samples  representing  the  Action 
and  TLV  levels,  respectively. 

The  theoretical  and  average  observed 
loadings  of  the  sample  tubes  for  each  study 
can  also  be  found  in  Tables  1  and  2.  Since 
EASC's  standard  generation  facility  has  just 
completed  modifications  to  permit  the  inde- 
pendent verification  of  the  contaminant 
loadings,  these  values  were  supplied  to  MAT 
1  and  2  participants  as  an  estimate  of  the 
sample  loadings.     Future  round-robin  samples 
will  have  a  more  definitive  number  attached 
to  them.     This  will  be  accomplished  by  anal- 
yzing the  contaminant  air  stream  as  the 
samples  are  being  prepared. 
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Table  1. 


Participant's  Results,  MAT  Round  1 
Acrylonitrile  on  Porapak  N  Tubes 


AN  Reported,  yg 

Sample         Sample  Sample 
Laboratory  X  Y  Z  Blank 


1 

6 

69 

6 

51 

13 

19 

0 

01 

2 

5 

75 

6 

0 

12 

25 

0 

08 

3 

4 

4 

4 

2 

8 

9 

0 

4 

1 

675* 

4 

854 

10 

46 

0 

027 

5 

4 

33 

4 

45 

9 

37 

0 

01 

6 

6 

68 

6 

61 

13 

36 

0 

00 

X           =     5.50  Xf  .   =  11.2 

(xSy)  UJ 

a=1.06  a=1.9 

rel.  a  =  19.3%  rel.  a  =  17% 

Summary  of  supporting  information  for  samples: 

X5Y  Z 

Theoretical  loading  (yg)  6.94  13.74 

EASC  average  observed 

loading  (yg)  5.92±0.34+  11.26±1.34t 

t99%  confidence  interval  based  on  the  analysis  of  9  tubes 
*data  point  omitted  from  calculation  of  mean;  rejection 
based  on  Q  test  at  90%  confidence  level 

Table  2.     Participant's  Results,  MAT  Round  2 
Acrylonitrile  on  Charcoal  Tubes 

AN  Reported,  yg 

Sample         Sample         Sample  Blank 
Laboratory         X  Y  Z 


1 

26 

62 

26 

39 

62 

15 

00.00 

2 

35 

1 

35 

8 

82 

4 

0.4 

3 

40 

40 

95 

ND 

4 

29 

27 

63 

5 

35 

5 

42 

9 

79 

3 

ND 

6 

31 

6 

31 

5 

77 

5 

0.0 

7 

39 

0 

39 

8 

89 

6 

000 

8 

50 

3 

48 

5 

110 

0 

0.0 

9 

33 

5 

33 

6 

78 

0 

ND 

10 

18 

22 

20 

04 

38 

71 

00.00 

X,  . 

=  34 

22 

X 

77 

57 

(z)  = 

a 

=  8 

38 

o  = 

19 

68 

rel 

a 

=  24 

5% 

rel 

.  o  = 

25 

4% 

Summary  of  supporting  information  for  samples: 

X&Y  Z 

Theoretical  loading  (yg)        43.36  99.91 

EASC  average  observed 

loading  (yg)  40.71±0.88*  98.14±1.18* 

ND  -  not  detected 

*99%  confidence  interval  based  on  analysis  of  24  tubes 
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LABORATORY  EVALUATION  TECHNIQUES  -  US  ARMY 
CALIBRATION  PROGRAM 


CHARLES  B.  COULTER 

US  ARMY  METROLOGY  AND  CALIBRATION  CENTER 
US  ARMY  MISSILE  COMMAND 
REDSTONE  ARSENAL,  AL  25809 


The  US  Army  utilizes  three  principal  techniques  in  the  evaluation 
of  its  calibration  laboratories:     Technical  Measurement  Audits,  Tech- 
nical Inspections  and  Certification  of  Technicians.     Each  major  cali- 
bration laboratory  is  subject  to  each  of  the  above  techniques  or  a 
combination  thereof  contingent  upon  its  mission  and  location  within  the 
Department  of  Army  logistical  complex.     Each  of  the  techniques  is  ad- 
ministered or  managed  by  the  US  Army  Metrology  and  Calibration  Center 
(USAMCC)  at  Redstone  Arsenal,  Alabama. 

Technical  Measurement  Audits  are  objective  in  nature  while  Technical 
Inspections  and  Certification  of  Technicians  rely  heavily  on  subjective 
considerations.     The  combination  establishes  a  comprehensive  quality 
assurance  program  for  calibration. 

Key  words:     Audit;  calibration;  calibration  laboratory;  certification; 
evaluation;  inspection. 


1.  Introduction 

US  Army  calibration  laboratories  are 
operated  in  accordance  with  regulatory  re- 
quirements and  technical  guidance  provided 
from  higher  echelon  activities.     The  regula- 
tory requirements  and  technical  guidance  as- 
sure, in  theory,  that  the  calibration  labor- 
atories perform  at  a  level  sufficient  to 
meet  mission  requirements.     In  practice,  ad- 
herence to  regulatory  requirements  and/or 
higher  echelon  technical  guidance  does  not 
in  itself  assure  quality  performance.  Addi- 
tionally, laboratory  personnel  are  always 
subject  to  ignore  the  requirements  and  guid- 
ance.    In  order  to  assure  satisfactory  per- 
formance, the  US  Army  periodically  evaluates 
its  laboratories,  utilizing  principally 
three  different  techniques:     Technical  Mea- 
surement Audits,  Technical  Inspections,  and 
Certification  of  Technicians.     The  tech- 
niques employed  are  used  to  confirm  that  the 
laboratories  are  capable  of  and  perform  at  a 
level  sufficient  to  fulfill  mission  require- 
ments . 


There  is  no  attempt  to  use  any  of  the 
aforementioned  techniques  to  rank  or  rate 
the  laboratories  against  one  another.  The 
primary  intent  is  to  confirm  performance  at 
a  predetermined  level.     It  constitutes  a 
pass-fail  evaluation. 

The  primary  responsibility  for  the 
establishment,  performance,  and/or  monitor- 
ing of  the  evaluation  processes  rests  with 
the  US  Army  Metrology  and  Calibration  Center 
(USAMCC )  at  Redstone  Arsenal,  Alabama.  The 
USAMCC  is  tasked  with  program  management  and 
technical  guidance  for  the  total  US  Army 
Calibration  System.     The  evaluations  serve 
as  the  means  to  determine  the  effectiveness 
of  the  program  management  and  the  validity 
of  the  technical  guidance. 

2.     Technical  Measurement  Audits 

In  essence,  a  Technical  Measurement 
Audit  is  a  "hands  on"  performance  test  of  a 
calibration  laboratory's  ability  to  provide 


precision  measurements.    Unless  problems  are 
encountered,  audits  are  conducted  entirely 
via  mail.     The  USAMCC  provides  a  primary  or 
reference  lab  standard,  with  the  certified 
value  unknown  to  the  audited  laboratory,  to 
the  audited  laboratory  for  calibration.  The 
value  measured  by  the  laboratory  is  then  com- 
pared to  the  primary  certified  value  of  the 
standard.     The  emphasis  is  on  the  accuracy 
and  precision  of  measurement.  Management 
and  administrative  practices  are  of  periph- 
eral concern  only. 

Parameters  currently  maintained  in  the 
audit  program  are:     DC  Voltage,  AC  Voltage, 
Resistance,  Capacitance,  Attenuation,  VSWR, 
Microwave  Power,  Absolute  Pressure,  Gage 
Pressure,  Length,  and  Mass.     The  standards 
for  the  various  parameters  are  supported  by 
the  US  Army  Standards  Laboratory  (ASL),  the 
Army's  primary  level  calibration  laboratory, 
located  at  the  USAMCC.     There  are  no  rigid 
intervals  of  calibration.     The  audit  stan- 
dards are  calibrated  as  deemed  necessary. 

Each  audit  package  is  composed  of  no 
more  than  one  DC  and  low  frequency,  one 
microwave  and  one  physical  parameter.  (For 
the  remainder  of  this  paper,  audit  package 
refers  to  the  hardware-audit  primary  or  sec- 
ondary reference  standards;  and  accompanying 
forms  and  instructions  shipped  to  the  audited 
facilities.)     The  audit  packages  are  tailored 
to  the  capability  of  the  audited  facility. 
Upon  receipt  of  the  package,  each  facility 
is  allowed  ten  working  days  to  perform  the 
measurements  at  the  points  specified  and 
place  the  audit  package  and  report  of  mea- 
surement in  the  mail  to  the  USAMCC.  The 
report  of  measurements  must  include  all  raw 
data,  manner  of  reduction  (computations), 
calibration  procedure  (to  include  traceabil- 
ity  of  laboratory  standards)  and  name  of  the 
technician  peforming  the  audit. 

Upon  receipt  of  the  report  of  measure- 
ment at  the  USAMCC,  an  analysis  of  the  re- 
sults is  performed.     The  analysis  addresses 
the  absolute  difference  between  the  mean 
value  of  the  standard  as  measured  by  the 
audited  laboratory  and  the  certified  value 
as  provided  by  the  ASL,  and  statistically 
measures  the  dispersion  (standard  deviation) 
about  the  mean  value  provided  by  the  audited 
laboratory.     If  both  the  mean  value  and  stan- 
dard deviation  are  judged  acceptable  and  the 
administrative  requirements  of  the  audit 
have  been  met,  the  audited  laboratory  is 
notified  by  letter  and  no  further  action  is 
taken.     Because  US  Army  laboratories  utilize 
US  Army  Technical  Bulletins  describing  step 
by  step  calibration  procedures,  little  ef- 
fort is  expended  toward  analysis  of  the 


procedure  unless  unsatisfactory  results  ar> 
obtained. 

In  the  event  that  an  audited  labora- 
tory's results  are  unsatisfactory  (either  j| 
mean  value  or  standard  deviation),  that 
laboratory  will  be  required  to  cease  suppo:| 
in  the  affected  parameter  or  downgrade  the:; 
capability  until  the  source  of  error  is  lo- 
cated and  the  conditions  surrounding  that  .] 
error  rectified.     Initial  effort,   (joint  bt.j 
tween  the  facility  and  the  USAMCC )  as  a  re-,j 
sultant  of  unsatisfactory  performance,  is  j 
directed  toward  cross  checking  of  the  audi"'] 
facility's  standards  within  house,  reeompu-j 
tation  of  data,  recertif ication  of  audit 
standards  and  confirmation  that  the  proper  i 
procedure  was  followed  in  total.     In  gener;- 
the  initial  effort  locates  the  source  of 
problem.     If  the  problem  cannot  be  isolatec 
to  drift  in  a  standard  (audit  or  laboratory- 
standard),  computational  error,  or  error  o: 
omission  in  the  procedure,  then  the  effort 
to  locate  the  problem  is  expanded  to  incluc 
examination  of  the  technical  correctness  ol 
the  procedure,  actual  capability  of  the  lal 
oratory's  standards,  training  of  the  techn^ 
cian,  competence  of  the  technician  or  satis, 
factory  support  of  the  laboratory's  standai; 
by  the  supporting  activity.     Concurrent  wi1 
the  attempt  to  isolate  the  problem,  the 
audited  activity  must  analyze  potential  im- 
pact upon  supported  items  caused  by  such 
unsatisfactory  measurements.  Corrective 
action  is  taken  as  required  to  assure  va- 
lidity of  calibration  of  supported  equip- 
ment . 

Subsequent  to  the  location  of  cause  of 
the  problem,  a  reaudit  utilizing  different  t, 
serial  numbered  audit  standards  is  conducts 
to  confirm  alleviation  of  the  problem.  Ree 
sons  for  past  failures  in  the  audit  "run  th 
full  gamut".     Although  rare,  the  reason  for 
failure  in  an  audit  is  sometimes  never  as- 
certained.    Extensive  investigation  leads  t 
no  conclusions.     If  a  reaudit  is  then  per- 
formed satisfactorily,  the  laboratory  re- 
sumes full  operations. 

Upon  conclusion  of  an  audit,  the  com-  ; 
mander  of  the  installation  at  which  the  latj 
oratory  is  located  is  provided  a  report  by  ■ 
letter  from  the  USAMCC.     The  report  provide 
quantitative  results  (difference,  standard 
deviation),  a  finding  of  AGREEMENT/NO  AGREE 
MENT  (satisfactory  or  unsatisfactory)  for 
each  parameter  audited,  complete  discussion 
(to  include  corrective  action)  or  any  prob- 
lems encountered  and  comments  concerning  a 
facility's  ability  to  react  to  and  rectify 
problems  in  the  event  of  a  NO  AGREEMENT 
finding. 
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Every  effort  is  made  to  convince  lab- 
's oratory  personnel  that  audits  are  for  their 
benefit  as  well  as  a  test  of  their  ability. 
Excluding  computational  errors  as  a  source 
of  failure,  there  have  probably  been  as  many 
failures  due  to  conditions  external  to  the 
audited  laboratory's  control  as  there  have 
c  been  directly  attributable  to  the  audited 
!  laboratory.     When  personnel  view  the  audit 
(  in  this  manner  and  perform  it  as  routinely 

as  a  normal  calibration  then  the  results 
t  truly  reflect  the  ability  of  that  laboratory 
:  to  meet  its  mission  requirements.     In  reali- 
ty, because  of  potential  consequences  of  a 
"I  failure,  a  significant  number  of  laboratories 
i  over-emphasize  the  audit  by  utilizing  only 
I  certain  technicians,  increasing  the  number 
1  of  measurements,  utilizing  multiple  methods 
"j  and  increasing  time  spent.     While  an  audit 
:  in  this  instance  can  check  standards,  pro- 
■:  cedure,  certain  personnel,  etc.,  it  does  not 
I  verify  the  everyday  quality  of  the  labora- 
j  tory. 

i  Technical  measurement  audits  are  con- 

:|  ducted  quarterly  at  eleven  Army  Area  Cali- 
'■  bration  Laboratories  (responsible  for  geo- 
-  graphical  area  support),  semiannually  at 
|  twenty-three  Army  Internal  Calibration  Lab- 
'i  oratories  (responsible  for  single  installa- 
i  tion  support )  and  annually  at  the  WATS  Sup- 
1  ply  Center,  Luxembourg  (as  a  courtesy). 
I 

3.     Technical  Inspections 

Technical  inspections  are  on-site  in- 
spections of  installation  calibration  pro- 
grams.    They  differ  from  audits  in  their 
scope  and  the  fact  that  they  encompass  lab- 

i  oratory  customers  as  well  as  the  laboratory. 

|  While  audits  are  based  almost  exclusively  on 
quantitative  or  objective  judgments,  inspec- 

;  tions  require  a  significant  degree  of  sub- 

'  jective  evaluation. 

The  USAMCC  conducts  technical  inspec- 
tions annually  at  the  ASL,  six  Continental 
US  (CONUS)  Area  Calibration  Laboratories, 
twenty-five  CONUS  Internal  Calibration  Facil- 
ities, approximately  one-third  of  the  fifty- 

■  two  Transfer  Teams  in  CONUS,  and  contractor 
facilities  as  the  need  arises.     In  addition, 

j1  the  USAMCC  has  been  tasked  to  expand  the 
Technical  Inspections  to  include  calibration 
activities  in  Europe  and  the  Pacific  Theater. 

Each  inspection  team  is  composed  of  one 
civilian  team  chief  and  two  to  four  civilian 
|  and/or  military  team  members  .  Technical 

■  (hardware)  expertise  is  usually  provided  by 
Senior  NCO's.     Inspections  require  from 
three  to  five  days  each,  dependent  upon  size 
of  the  installation  to  be  inspected. 


Transfer  teams,  as  small  mobile  operations, 
are  generally  inspected  in  one  to  two  days 
by  two  Senior  NCO's. 

Technical  inspections  address  two  prin- 
cipal areas:     (l)  The  capability  of  the  in- 
stallation or  activity  to  accomplish  its  as- 
signed calibration  mission  to  include  the 
availability  of  technical  skills,  technical 
information,  measurement  methods  and  accuracy 
of  measurements,  and  (2)  Compliance  with  pre- 
scribed policies  and  procedures.  Specifi- 
cally the  inspection  team  reviews:     (l)  Ad- 
ministrative and  management  practices,  (2) 
Availability,  maintenance  and  use  of  current, 
approved  calibration  procedures,   (3)  Control 
of  forms,  records,  reports  and  applicable 
procedures,   (k)  The  internal  quality  assur- 
ance surveillance  program,   (5)  Control, 
maintenance  and  operation  of  the  calibration 
recall  system,   (6)  Environmental  and  safety 
controls,   (T)  Knowledge  of  and  compliances 
with  appropriate  references,   (8)  Training 
program,   (9)  Maintenance  practices,  and  (10) 
Technical  capability  of  calibration  techni- 
cians in  the  maintenance  and  use  of  measure- 
ment standards,  use  of  hand  tools,  use  of 
calibration  procedures,  and  performance  of 
calibration . 

The  most  time  consuming  facet  of  the 
technical  inspections  (other  than  time  re- 
quired visiting  customer/owner  areas)  is  the 
review  of  the  technical  capability  of  the 
calibration  technicians.     This  is  accom- 
plished through  "end-item  checks."  Cali- 
bration technicians  perform  a  "hands  on" 
calibration  for  validity  of  calibration  of 
an  item  recently  calibrated  while  an  inspec- 
tion team  member  observes  the  performance 
"over  the  shoulder."     Such  a  check  encom- 
passes many  of  the  ten  specific  areas  of  re- 
view listed  previously,  i.e.,  use  of  approved 
calibration  procedures,  use  of  proper  forms, 
observance  of  proper  safety  practices,  etc. 
as  well  as  the  technical  capability  of  ex- 
pertise of  the  technician. 

Inspection  findings  are  divided  into 
two  categories:     Deficiencies  and  Observa- 
tions.    Deficiencies  are  shortcomings  that 
affect  the  technical  capability  of  the 
laboratory,  safety  or  health  hazard  or  con- 
ditions that  if  allowed  to  continue  would 
lead  to  technical  problems  or  health  or 
safety  hazards.     Observations  indicate  con- 
ditions that  are  contrary  to  prescribed 
regulations  and  policy  or  good  management 
practice  but  would  not  be  of  sufficient  im- 
pact as  to  be  classified  a  deficiency.  As 
the  findings  are  made  during  the  inspection, 
every  effort  is  made  by  the  inspection  team 
to  assist  in  alleviating  the  shortfall  or 
initiating  requisite  action. 


Selected  activities  receive  an  overall 
rating  of  satisfactory  or  unsatisfactory  at 
the  conclusion  of  the  inspection  based  upon 
severity  of  observations  and  deficiencies. 
An  unsatisfactory  rating  may  be  the  basis 
for  discontinuing  calibration  operations  un- 
til the  adverse  conditions  are  corrected. 
Satisfactory  ratings  may  be  accomplished  by 
directed  corrective  action,  while  unsatis- 
factory ratings  necessitate  a  reinspection. 

An  unsatisfactory  rating  may  be  given 
when  any  of  the  following  adverse  conditions 
are  noted:     (l)  An  invalid  calibration  of  a 
measurement  system,   (2)  A  safety  or  health 
hazard,   (3)  Damage  to  measurement  standards, 
TMDE  or  property,   (U)  An  adverse  impact  on 
the  combat  effectiveness  of  a  weapon  system 
or  tactical  organization  due  to  calibration 
support,   (5)  Invalid  calibration  of  a  criti- 
cal nuclear  weapons  TMDE,   (6)  Calibration  of 
TMDE  or  measurement  standards  when  not  in  the 
prescribed  environment,  and  (T)  Loss  of 
traceability  to  national  standards. 

At  the  conclusion  of  the  inspection, 
the  commander  of  the  installation  or  activi- 
ty is  informed  of  the  findings.     Upon  return 
to  the  USAMCC,  the  inspection  team  prepares 
and  forwards  a  comprehensive  report,  includ- 
ing required  corrective  action  when  appropri- 
ate, to  the  installation  commander.  The 
commander  in  turn  outlines  corrective  action 
taken  or  to  be  taken.     The  next  inspection 
will  confirm  whether  or  not  the  corrective 
action  was  accomplished. 

h.     Certification  of  Technicians 

Certification  of  technicians  is  a  rela- 
tively new  program  and  is  currently  restrict- 
ed in  its  application.     Calibration  techni- 
cians are  encouraged  to  voluntarily  enroll 
in  a  skill  certification  program  consisting 
of  a  combination  written  and  "hands  on"  per- 
formance examination  to  verify  their  skills. 

Each  participating  installation  is  re- 
sponsible for  developing  and  maintaining  a 
plan  for  certifying  its  calibration  techni- 
cians.    The  installation  in  a  joint  effort 
with  the  USAMCC  establishes  a  set  of  stan- 
dards that  are  definitive  and  contain  the 
essential  elements  in  sufficient  detail  to 
adequately  evaluate  the  individual  techni- 
cian's performance. 

The  standards  stress  three  basic  as- 
pects:    (l)  Core  knowledge,   (2)  Speciality 
knowledge,  and  (3)  Demonstrated  ability. 
Core  knowledge  addresses  the  basic  skills 
and  knowledge  necessary  to  perform  in  a 
functional  area,  i.e.,  interpretation  of 


drawings  and  schmatics,  use  of  basic  test 
equipment,  etc.     Specialty  knowledge  refei 
to  the  specialized  skills  and  knowledge 
necessary  to  perform  duties  as  stated  in  t 
job  description.     Demonstrated  ability  is 
the  "hands  on"  application  of  core  and  spe 
cialized  knowledge. 

The  actual  evaluation  for  certificati 
consists  of  a  written  examination  (prepare 
by  the  USAMCC)  encompassing  both  core  and 
specialized  knowledge,  a  "hands  on"  profic 
ency  examination  administered  by  a  compete 
individual  or  individuals  and  an  experienc 
profile  that  clearly  identified  and  compar 
a  technician's  actual  experience  and  train 
ing  with  the  program  requirements.  The  re 
suits  of  the  evaluation  are  reviewed  by  th 
installation's  Certification  Review  Commit 
tee  who  in  turn  recommend:  (l)  Approval  0 
certification  of  the  technician,  (2)  Furth 
development  and  training  of  the  technician 
or  (3)  Withdrawal/reinstatement  of  the  cer 
fication  of  the  technician. 

Current  policy  dictates  a  three  year 
interval  for  recertif ication  of  a  technici 
who  routinely  performs  within  the  area  for 
which  certification  was  awarded.     If  the 
technician  is  reassigned  to  another  area  03 
does  not  routinely  perform  in  that  area, 
then  a  two  year  interval  is  invoked. 

5 .  Summary 

The  laboratory  evaluation  techniques 
described  above  provide  a  comprehensive  ex 
amination  of  the  ability  of  US  Army  Cali- 
bration Laboratories  to  fulfill  their  mis- 
sion requirements.     Implementation  of  the 
techniques  requires  considerable  managerial 
or  program,  and  technical  expertise  and  a 
degree  of  common  sense.     The  techniques,  i 
employed  conscientiously  will  confirm  the 
ability/inability  of  an  army  laboratory  to 
perform  satisfactorily.     To  this  author's 
knowledge,  utilization  of  a  combination  of 
the  above  techniques  has  not  failed  to  vali 
date  the  status  (satisfactory  or  unsatisfac 
tory)  of  a  calibration  program  correctly. 
The  techniques  are  sound.     The  effectivene 
of  the  program,  as  with  any  quality  assur- 
ance program,  rests  with  management's  abil 
ty  or  desire  to  correct  detected  shortcom- 
ings . 
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Because  coal  is  heterogeneous,  special  procedures  are  necessary 
for  the  determination  of  the  quality.    Commercially,  the  amount  of 
coal  to  be  sampled  in  a  year  for  a  contract  may  be  a  million  tons. 
Both  the  seller  and  the  buyer  need  a  proven  system  for  contractual 
acceptance  of  the  determined  quality.    The  necessity  for  including 
both  the  automatic  sampler  and  the  laboratory  in  such  a  system  is 
discussed  and  the  resulting  split  sample  program  is  shown  to  be  a 
natural  development.    The  program  serves  the  primary  purpose  of 
determining  the  quality  but  it  also  serves  as  a  continuous  eval- 
uating system  of  the  in-house  laboratory  procedures. 

Key  words:    Heterogeneous;  in-house;  laboratory;  preparation; 
riffle;  sample;  standards. 


1 .  Introduction 

Almost  every  news  report  refers  to  coal 
as  /America's  "Energy  Ace  in  the  hole."  This 
major  energy  source  is  used  primarily  as  a 
fuel  for  the  generation  of  electricity.  In 
recent  years,  the  advancements  made  in  in- 
creasing efficiencies  in  utilizing  this  fuel 
have  been,  in  part,  due  to  the  construction 
of  larger  steam  generating  stations.  To 
supply  such  users  the  increased  tonnage  nec- 
essary, more  efficient  coal  handling  methods, 
also,  have  been  developed.    Unit  trains  with 
dedicated  equipment  hauling  as  much  as  ten 
thousand  tons  are  common.    The  mines  have  in- 
stalled facilities  for  rapid  loading  of  these 
trains,  often  loading  at  the  rate  of  five 
thousand  tons  per  hour.    The  generating  sta- 
tions have  installed  facilities  for  rapid 
unloading  of  these  trains  and  some  installa- 
tions can  unload  such  a  ten  thousand  ton 
train  in  less  than  an  hour. 

But  coal  is  a  most  heterogeneous  fuel 
and  special  attention  and  special  equipment 
is  necessary  for  the  correct  sampling.  Con- 
tracts requiring  the  delivery  of  a  million 
tons  a  year  are  common  and  these  contracts 


usually  include  a  premium/penalty  quality 
clause  guarantee.    For  such  a  clause  to  be 
meaningful,  both  sampling  equipment  operation 
and  the  laboratory  procedures  must  have  con- 
tractual acceptance  by  both  the  supplier  and 
user.    The  split  sample  program  with  in-house 
laboratory  analyzing  is  one  satisfactory  and 
successful  solution.    It  is  also  a  most  ex- 
acting and  demanding  program. 

2.    Sampling  System 

It  is  preferable  that  coal  be  weighed 
and  sampled  at  the  same  jpoint  in  time.  If 
there  is  a  lapse  in  time  between  these  two 
events,  consideration  should  be  given  by 
both  the  purchaser  and  seller  to  changes  in 
moisture  during  this  interval  and  the  con- 
sequent shift  in  relationship  of  moisture  to 
the  true  quality  of  the  coal  at  the  instant 
when  ownership  of  the  coal  trans fered  from 
one  to  the  other  C ID-    The  standard  operating 
condition  is  for  the  train  to  be  weighed  at 
the  loading  site,  therefore  it  is  preferable 
for  the  sampling  to  be  done  at  the  loading 
site. 
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The  specifications  for  such  a  sampling 
system  are  normally  those  that  meet  the 
A.S.T.M.  Standards  D-2013,  "Preparing  coal 
samples  for  analysis"   C2J  and  D-2234,  "Col- 
lection of  a  gross  sample  of  coal"    [3  J.  It 
is  not  unusual  for  the  specifications  to  be 
such  that  even  more  increments  are  taken  than 
are  required  to  meet  these  standards.  The 
design  may  require  as  many  as  three  or  four 
sample  cutters  in  series.    This  is  necessi- 
tated by  the  large  primary  increment.    At  a 
5000  tons  per  hour  loading  rate  for  3"  x  0 
coal,  the  primary  increment  can  be  as  much 
as  thirty- three  (33)  tons  per  hour.    A  typi- 
cal three  stage  sampler  is  shown  in  Figure  1. 
Such  a  sampling  system  with  associated  con- 
veyors, feeders,  crushers,  etc.  is  a  large 
material  handling  installation  and  is  expen- 
sive. 

2.1    Sampling  System  Example 

If  a  coal  handling  installation  sampling 
3"  x  0  washed  coal  loading  at  the  rate  of 
5000  tons  per  hour  is  used  as  an  example, 
what  are  some  of  the  basic  components?  Such 
an  installation  requires  a  primary  cutter 
with  an  opening  of  9-inches  to  travel  at  the 
rate  of  18-inches  per  second  at  75-second  in- 
tervals.   Each  increment  is  1400  pounds. 
This  is  thirty-three  tons  per  hour. 

A  large  feeder  discharges  this  coal  in- 
crement to  a  secondary  cutter,  also  with  an 
opening  of  9-inches.    This  will  divide  the  in- 
itial increment  to  about  5000  pounds  per  hour. 

This  secondary  sampler  product  is  then 
crushed  to  a  top  size  of  number  8  standard 
sieve  size.    The  crusher  product  is  fed  to  a 
tertiary  sampler  with  an  opening  of  V%- inches. 
Even  with  three  sample  cutters  operating  in 
series,  the  tertiary  product  can  be  as  much 
as  30  kilograms  per  hour. 

3.  Sample  Preparation 

The  final  amount  of  product  from  the 
automatic  sampler  is  too  much  for  the  labor- 
atory and  the  amount  is  divided,  with  en- 
closed rifflers,  until  the  desired  sample  of 
about  1000  g  is  available   P4] .    There  are  usu- 
ally at  least  three  such  samples  retained: 
one  is  for  the  seller,  one  is  for  the  pur- 
chaser, and  one  is  retained  as  a  referee 
sample.    Each  of  these  is  immediately  sealed 
in  a  heavy  vapor-impervious  bag  and  properly 
identified. 

4.  Sample  Handling 

Prompt  movement  of  the  split  samples  of 

coal  to  the  laboratories  is  important.  Al- 
though the  coal  sample  is  sealed  in  a  heavy 


vapor- impervious  bag,  it  is  essential  that 
the  sample  be  analyzed  as  soon  as  possible 
because  deterioration  is  a  problem.    At  one 
time,  A.S.T.M.  Standards  referred  to  deteri 
oration  of  coal  samples  and  warned  of  time 
elapse  not  to  exceed  30  days  [53 .    This  has 
since  been  removed  from  the  standards  becau 
such  a  definite  time  implied  and  was  often  L 
assumed  that  the  deterioration  was  not  seri 
ous  for  these  30  days. 

5.  Sampling  System  Acceptance 

The  acceptance  of  an  automatic  samplin 
system  is  equally  important  to  both  the  sel 
and  the  buyer.  Both  recognize  the  apprecia 
economic  importance  of  its  use.  Both  are  d 
pendent  on  it  for  a  representative  sample. 
It  must  function  as  designed  and  installed. 
It  must  be  able  to  sample  a  whole  consignme 
at  approved  intervals.  It  must  function  in' 
manner  that  the  increments  and  the  resultin 
samples  are  free  of  contamination.  It  must 
be  dependable  as  an  operating  piece  of  equi 
ment. 

It  is  important  to  the  seller  because 
the  quality  of  the  product  must  be  known  be 
fore  it  can  be  offered  in  the  market  place. 
It  is  important  to  the  purchaser  because  th' 
quality  of  the  product  will  be  based  on  the 
split  sample.    Improper  operation  can  resul 
in  a  bias  sample  and  only  extensive  manual  jj 
sampling  may  determine  the  effect  on  the  sa 
pie.    Contracts  include  specific  statements 
allowing  for  observation  and  inspection. 

Such  a  method  for  procuring  a  split 
sample  then,  becomes  a  contractual  accepted 
procedure . 

6.  Analytical  Procedures 

The  analytical  procedures  followed  by 
the  laboratories  determining  the  quality  fbj 
contractual  acceptance  are  those  included  i: 
the  contract.    The  contract  may  state  simpl 
"All  sampling  and  testing  will  be  in  accor- 
dance with  current  applicable  A.S.T.M.  Stan 
dards  [6]  or  it  may  detail  specific  condi- 
tions that  must  be  considered  as  critical  i\ 
analyzing  for  full  compliance.    Other  recog 
nized  standards,  such  as  Bureau  of  Mines  an 
International  Organization  for  Standardiza- 
tion, are  also  designated  as  applicable. 

The  parameters  considered  important  foj 
the  particular  coal's  use  are  included  in  tfj 
contract  and  these  are  checked  by  the  labor 
tories.    The  results  are  empirical  and  are 
reproducible  by  carefully  adhering  to  defi- 
nite prescribed  methods  and  standards  [7J . 
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Laboratories  testing  the  same  coal  sam- 
,  pie  but  using  different  standards  may  report 
'i  results  that  do  not  agree.    Such  differences 
are  not  to  be  interpreted  as  a  fact  that  one 
laboratory  is  reporting  incorrect  results. 
*  Each  can  be  correct  for  the  method  and  stan- 
dard used.    Such  differences,  however,  are 
expected  to  be  constant.    Such  possible  dif- 
1  ferences  are  the  reasons  the  standard  to  be 
used  must  be  included  in  the  contract. 

7.    Laboratory  Acceptance 

The  criteria  for  acceptance  of  any  lab- 
.  oratory  as  reputable  is  difficult  to  define 
!  free  of  ambiguities.    The  personnel  must  be 
"  trained  and  experienced.    The  equipment  for 

the  tests  must  be  available  and  calibrated, 
i Precision  and  accuracy,  both,  must  be  ex- 
^  pec ted.    These  are  obvious  and  no  laboratory 
]  would  be  expected  to  be  accepted  without  such. 
,  Then  what  criteria  are  recognized  as  estab- 
;  lishing  contractual  acceptance  of  in-house 

laboratory  procedures  when  the  split  sample 

program  is  used? 

J         7.1    Representative  Sample 

The  split  sample  is  the  sample  repre- 
senting the  whole  consignment.    It  is  the 

|  sample  taken  by  the  automatic  sampler.  It 
is  the  sample  that  is  divided  without  human 
choice  in  its  division.    It  is  the  proper 

J  size  and  amount  for  the  laboratory. 

7.2  Sample  Preparation 

The  split  sample  will  be  prepared  in  an 
identical  manner  for  each  consignment. 

7 . 3  Routine  Testing 

The  number  of  such  samples  supplied  to 
i  each  of  the  two  in-house  laboratories  each 
j year  is  large.    If  the  average  train  weight 
^  is  10,000  tons,  then  one  hundred  samples  can 
_  be  expected  for  a  million  ton  a  year  contract. 
.  Each  sample,  then,  is  handled  and  tested  ac- 
cording to  the  standards  in  a  routine  manner. 

7.4  Continuous  Checking 

As  the  number  of  samples  in  the  program 
increases,  the  statistical  variations  per- 
mitted will  be  predictable.    Each  laboratory's 
J  results  will  serve  as  a  constant  check  on  the 
u  performance  of  the  other  laboratory. 

3 

7.5  Continuous  Cross-check 

Few  sellers  supply  coal  to  but  one  buyer 
and  few  buyers  receive  coal  from  but  one 
seller.    It  follows  then  that  there  is  a  con- 
stant cross-checking  of  in-house  laboratory 


performance  and  procedures  with  the  split 
sample  program. 

7.6    Criteria  for  Accreditation 

Such  is  the  criteria  for  contractual 
acceptance  of  in-house  laboratory  procedures. 
It  can  be  considered  as  volunteer  laboratory 
accreditation  by  contractual  compliance. 

8.  Precautions  for  a  Successful 
Split  Sample  Program 

8.1  Sampler  Agreement 

The  automatic  sampling  system's  specifi- 
cations and  design  should  be  agreed  upon  as 
acceptable  by  both  the  coal  buyer  and  the 
coal  seller.    Its  operation  will  meet  such 
specifications . 

8.2  Handling  Agreement 

The  sample  preparation  and  handling  of 
the  product  from  the  automatic  sampler  will 
be  under  the  supervision  of  trained  personnel 
and  will  be  performed  according  to  standards 
agreed  upon  by  both  the  coal  supplier  and 
coal  buyer. 

8.3  Sample  Delivery 

The  sample  will  be  dispatched  to  both 
laboratories  without  delay  in  the  most  effi- 
cient manner. 

8.4  Statistical  Analysis 

A  statistical  analysis  of  the  compara- 
tive results  as  well  as  interpretation  of 
variants  must  be  continuous. 

9.  Round  Robin  Testing 

The  fuel  laboratories  in  the  coal  in- 
dustry do  not  rely  on  just  split  samples  for 
checking.    Round  robin  samples  are  also  used. 
These  samples  are  specially  prepared  from  a 
single  "unknown"  sample.    Care  is  taken  to 
divide  the  original  sample  into  as  many  equal 
parts  as  needed  by  continually  mixing  and 
separating  the  particals.    Because  of  the 
heterogeneous  characteristics  of  coal,  the 
sample  is  normally  reduced  to  at  least  2.50 
millimeter  and  usually  to  an  even  smaller 
250  micron  size.    Such  crushing,  mixing  and 
separating  is  expected  to  loose  some  of  the 
original  total  moisture,  an  important  para- 
meter in  all  commercial  coal  quality  evalua- 
tions.   This  loss  does  not  negate  the  sample 
as  useful  for  checking  but  instead  it  becomes 
exactly  that,  a  sample  useful  for  verifying 
the  results  obtained  by  routine  laboratory 
procedures  between  comparing  laboratories. 


The  value  of  such  checking  of  routine  labora- 
tory procedures  decrease,  though,  if  special 
handling,  not  routine  handling,  is  initiated 
by  the  personnel. 

Special  handling  is  expected  to  be  used 
if  a  proposed  procedural  change  is  being  con- 
sidered or  new  equipment  is  being  standard- 
ized. 

A  standardized  sample  is  a  sample  with 
known  values  for  such  round  robin  testing. 

10.    Summary  and  Conclusions 

10.1  Sampling  is  more  than  just  a  basic 
adjunct  to  coal  testing. 

10.2  A  split  sample  representing  the 
coal  at  the  moment  when  the  ownership  trans- 
fers from  the  supplier  to  the  buyer  at  the 
time  of  weighing  is  the  only  sample  that  re- 
presents the  true  quality  purchased. 

10.3  A  program  of  continuous  analysis 
of  split  samples  will  serve  as  a  program  of 
continuous  monitoring  of  procedures  between 
laboratories.  Such  a  program  promotes  rou- 
tine treatment  of  all  samples. 

10.4  Round  robin  samples  are  essential 
for  establishing  technics  and  standardizing 
equipment  but  the  split  sample  program  is 
essential  for  establishing  the  quality. 

10.5  As  the  number  of  such  split  sample 
agreements  increases  for  the  same  supplier  or 
for  the  same  buyer,  each  in-house  laboratory 
is  more  recognized  for  its  professional  com- 
petency because  of  the  increase  in  opportun- 
ities for  multiple  acceptance. 

10.6  The  value  of  split  sample  programs 
for  contractual  acceptance  of  in-house  labor- 
atory procedures  is  that  it  shows  that  such  a 
program  must  be  recognized  as  a  logical  cri- 
teria for  continuous  satisfactory  laboratory 
accreditation  for  purpose  of  commerce. 
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SUMMARY  OF  EVALUATIONS  OF  CLINICAL  LABORATORIES  PARTICIPATING  IN 
THE  CENTER  FOR  DISEASE  CONTROL  EVALUATION  PROGRAM 

LOUIS  C.   LAMOTTE,  JR.,  SC.D. 

LICENSURE  AND  PROFICIENCY  TESTING  DIVISION 
BUREAU  OF  LABORATORIES 
CENTER  FOR  DISEASE  CONTROL 
ATLANTA,  GEORGIA  30333 

During  the  period  from  1969  to  1979,  the  Center  for  Disease 
Control  has  provided  for  the  evaluation  of  about  1,000  interstate 
clinical  laboratories  for  licensure  purposes,  and  has  provided  a 
performance  evaluation  program  to  an  additional  1,700  clinical 
laboratories  that  were  not  required  to  participate  by  the  CLIA  '67 
law.     Licensed  laboratories  are  required  to  use  personnel  who  meet 
personnel  standards,  to  meet  applicable  standards  for  internal 
quality  control  of  tests,  and  to  perform  satisfactorily  in  various 
proficiency  testing  programs.     In  most  cases  the  personnel  and 
internal  quality  control  were  not  evaluated  for  nonlicensed 
participants.     A  review  of  the  data  suggests  that  some  labora- 
tories meet  or  exceed  most  of  the  standards,  but  that  a  disturbing 
number  of  laboratories  do  not.     The  demonstration  of  specific  poor 
performance  is  often  a  sufficient  stimulus  to  improvement;  in  some 
cases  regulatory  authority  seems  necessary  to  bring  about  improve- 
ment.    This  paper  will  provide  data  based  on  the  CDC  program. 


Key  words:     Clinical  Laboratories  Improvement  Act  of  1967;  CLIA  '67; 
Clinical  laboratory  evaluation;  Evaluation,  qualifications  of  labora- 
tory personnel;  Evaluation,  internal  quality  control;  Evaluation, 
performance;  Checklist;  Proficiency  testing;  Improvement. 


1 .  Introduction 

From  1969  through  1978,  the  Center  for 
isease  Control   (CDC)  conducted  a  clinical 
aboratory  evaluation  and  improvement  pro- 
ram  as  a  part  of  its  responsibility  for 
icensing  laboratories  operating  in  inter- 
tate  commerce.     The  950  interstate 
aboratories  were  expected  to  meet 
tandards  for  personnel,  for  internal 
uality  control  systems,  and  for  perform- 
nce.     Approximately  1,800  additional 
aboratories  participated  as  nonlicensed 
olunteers  in  the  performance  evaluation 
art  of  the  CDC  program.     As  of  1978,  the 
ealth  Care  Financing  Administration  (HCFA) 
fas  given  regulatory  responsibility  for 


licensing  interstate  laboratories  as  well 
as  for  certifying  for  payment  those  labor- 
atories that  serve  Medicare  patients;  CDC 
continues  to  evaluate  performance  of  non- 
licensed  participants.     In  addition,  CDC 
now  is  responsible  for  conducting  an 
independent  assessment  of  the  HCFA  evalua- 
tion system  and  surrogate  systems,  offering 
advice  and  constructive  criticism  where 
needed . 

The  purpose  of  this  paper  is  to 
describe  the  methods,  selected  observations, 
and  impact  of  the  evaluation  and  improve- 
ment program  operated  by  CDC  from  1969 
through  1978. 
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2.  Materials  and  Methods 

There  are  standards  currently  appli- 
cable for  clinical  laboratory  personnel 
which  specify  minimal  qualifications  for  the 
laboratory  director,  technical  supervisor, 
general  supervisor,  and  technologists. 
These  standards  indicate  education, 
experience  and  training  requirements  for 
each  position  category.1     Personnel  submit 
information  along  with  an  annual  appli- 
cation for  licensure  which  is  verified  by  an 
in-house  study  before  a  license  is  issued  to 
the  laboratory. 

There  are  also  standards  currently 
applicable  for  a  quality  control  system  for 
clinical  laboratories.     The  quality  control 
system  includes  procedures  covering  a  range 
of  items  from  equipment  calibration  and 
maintenance  to  validation  of  methods, 
surveillance  of  results,  remedial  action  for 
defects,  written  procedures  manuals, 
records,  and  specimen  handling  and  track- 
ing.2   A  detailed  checklist  is  used  by  CDC 
Examiners  to  assure  that  each  laboratory's 
quality  control  system  is  uniformly 
examined.     The  importance  of  this  checklist 
in  assuring  an  objective  and  complete  on- 
site  evaluation  cannot  be  over-emphasized. 
A  copy  of  one  page  from  a  CDC  checklist  is 
shown  in  Attachment  1 . 

Proficiency  test  samples  for  assessing 
a  laboratory's  performance  are  prepared  and 
distributed  by  CDC  in  the  categories  of 
microbiology,  dianostic  immunology, 
clinical  chemistry,  pharmacology,  and  toxi- 
cology. ^     Figure  1  shows  some  of  the  samples 
which  are  mailed. 

Performance  of  participants  is  assessed 
for  summary  purposes  in  relation  to  the  per- 
formance of  reference  laboratories. 
Generally,  participants  in  the  qualitative 
tests  are  assessed  in  terms  of  the  percent- 
age that  reported  the  correct  identification 
of  each  of  the  test  organisms;  in  quantita- 
tive tests,  the  parameters  provided  by  the 
reference  laboratories  are  used  as  the 
targets  for  participants.     The  number  of 
clinical  laboratories  that  received  samples 
in  1978  is  shown  in  Table  1. 

3.  Results  and  Program  Impact 

As  an  example  of  the  impact  of  the 
program  Figure  2  shows  the  progressive 
degree  of  adoption  of  a  particular  standard 
by  172  laboratories  in  the  CDC  program  from 
1969  through  1976.     Note  that  at  the  outset 
only  about  half  of  the  laboratories  had  an 
acceptable  preventive  maintenance  program; 


by  1976  95%  had  a  routine  program.  Figure 
shows  the  percent  of  bacteriology  labora-  L 
tories  that  were  checking  their  media  in 
1969,  and  the  progressive  improvement  over  •(.-: 
time.     Figure  4  shows  that  the  more  recent!  .- 
enrolled  laboratories  as  a  group  adopted  J 
this  bacteriology  quality  control  standard  . 
promptly  with  less  necessity  for  reiter- 
ation by  CDC  examiners. 

I  ill 

Another  analysis  of  program  impact  is', 
afforded  by  the  data  presented  in  Table  2.  V 
The  percentages  of  laboratories  licensed 
in  1971   (the  class  of  1971)  that  met 
several  selected  quality  control  standards  j 
for  each  year  from  1971  through  1977  are 
shown.     For  comparison,  the  data  for  the 
more  recently  enrolled  class  of  1977  are 
also  presented.     Obviously,  there  was  some];; 
improvement  in  the  degree  of  adoption  of 
these  quality  control  standards. 

In  Figure  5,  the  performance  of  all 
participants  enrolled  in  bacteriology  is 
"shown,   regardless  of  when  particinating 
laboratories  came  into  the  program.'  The 
newest  enrollees  will  have  little  or  no 
experience  with  the  program,  whereas 
others  will  have  had  the  benefit  of  pro- 
gram stimulation  for  years.  Because 
samples  in  each  shipment  are  not  identical[ 
the  challange  varies  from  shipment  to 
shipment.     However,  the  standard  test 
organisms  are  recycled  regularly  so  in  eve; 
year  participants  will  have  at  least  sever1 
opportunities  to  isolate  and  identify  the 
most  important  pathogens.     The  results  indi 
cate  that  in  most  years  from  10  to  20%  or 
more  of  the  participants  make  grades  of  lei, 
than  75  on  the  shipment  consisting  of  five 
or  more  samples  and  as  a  consequence  their^  ] 
efforts  often  reflect  their  best  performan, 
this  measurement  suggests  that  efforts  to 
stimulate  improvement  should  be  continued.  ' 


An  abstract  of  the  qualification 
requirements  for  clinical  laboratory 
personnel   (CLIA  '67)  can  be  obtained  from  ' 
the  author. 

2  .        )i  -t- 

A  summary  of  the  standards  for  in- 
ternal quality  control   (CLIA  '67)  can  be    1  ': 
obtained  from  the  author. 

3 

A  description  of  the  proficiency  tee  i* 
program  and  shipping  schedule  can  be 
obtained  from  the  author. 

'  Hi; 
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Evidence  of  improvement  in  bacteriology 
presented  in  Table  3.     Eleven  laboratories 
hat  entered  the  program  in  1972  in  bacteri- 
logy  were  followed  through  1978.     Only  7 
ffered  services  in  gonorrhea  diagnosis  in 
972,  and  of  these,  3  performed  satis- 
actorily  —  a  pattern  that  has  persisted 
hrough  1978. 

The  degree  of  improvement  is  obviously 
elated  to  how  poor  performance  was  when 
he  improvement  stimulus  was  initially 
bplied.     Figure  6  shows  that  laboratories 
testing  relatively  few  specimens  for 
jionorrhea  performed  the  poorest  but 
'enefited  most  from  the  program  stimulus. 
IP  contrast,  the  high-volume  and  larger 
aboratories  performed  so  well  initially  that 
Tiprovement  is  much  more  difficult  to 
emonstrate . 

Laboratory  performance  in  detecting 
epatitis  B  surface  antigen  in  sera  is  shown 
in  Table  4.     The  progressive  abandonment  of 
he  less  sensitive  second-generation  test  is 
indicated;  this  was  stimulated  by  the  pro- 
ram's  demonstration  that  the  test  was  less 
nsitive  than  the  third  generation  test. 

The  difference  in  variances  between 
ests  for  rheumatiod  factor  before  and  after 
ne  program  provided  standardization  is  shown 
n  Figure  7.     Note  that  this  program  effort 
suited  in  a  considerable  improvement  from 
very  wide  variance  between  laboratories  to 
variance  of  only  0.93. 
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Program  stimuli  have  a  multiplier  effect 
n  most  disciplines.     For  example,  demon- 
tration  of  poor  performance  can  be  expected 
b  stimulate  individuals  to  improve  their 
erformance  and  to  help  the  staff  justify 
jbstituting  more  accurate  and  precise 
rocedures  for  archaic  procedures.     A  case 
n  point  is  the  erythrocyte  count  (Figure  8); 
ip  laboratorians  have  improved  their  indivi- 
dual performance,  they  have  also  replaced 
ne  less  reliable  procedures  with  more 
bcurate  and  precise  procedures.  We 
p  commend  as  a  guiding  principle  "If  it's 
brth  doing,  it's  worth  doing  well." 

The  final  example  of  program  impact 
oncerns  the  stimulating  effect  of  a  notice 
o  a  laboratory  that  its  license  is  to  be 
evoked  because  of  poor  performance.  A 
tudy  of  grades  laboratories  made  in 
acteriology  for  five  shipments  before  and 
jive  shipments  after  receiving  such  notices 
nows  that  their  average  grades  increased 
o  8.9  points.     Concurrently,  the  average 
rades  of  a  randomly  selected  sample  of 
aboratories  that  did  not  receive  a  notice 


declined  1.4  points.     This  suggests  that 
laboratories  that  are  not  motivated  by 
the  simple  demonstration  of  poor  perform- 
ance are  motivated  by  the  threat  of 
license  revocation. 
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MATERIALS  CFR  74.20(c)  405.1317  (a)(3) 

14.    The  following  are  labeled  (identity,  titer  Antimicrobials 
or  concentration,  recommended  Antisera 
storage,  preparation  or  expiration  date)              Control  Organisms 

Media  (tubes,  plates  kits) 
Disc 

NO  (Code) 

COMMENT 

□  01080 

□  01081 

□  01082 

□  01083 

□  01084 

Strips 

Reagent  Solutions 

U  OlOob 
□  01086 

15.    Materials  in  use  are  "in-date",  reactive,  Antimicrobials 
and  not  deteriorated  Antisera 

Control  Organisms 
Media  (tubes,  plates,  kits) 
Disc 

□  01087 

□  01088 

□  01089 

□  01090 

□  01091 

Strips 

Reagent  Solutions 

□  01092 

□  01093 

TESTING  OF  REAGENTS  CFR  74.20(a)  405.1317(a)(1)  405.1317(b)(1) 

16.    The  following  reagents  (in  use  as  individual 
biochemical  tests  or  as  a  part  of  a  commercial 
system)  are  tested  each  day  of  use  to  Catalase 
demonstrate  a  positive  and  negative  Coagulase 
biochemical  reaction                                  Ferric  Chloride 

Indole 

Positive 
Reaction 
NO  (Code) 

Negative 
Reaction 
NO  (Code) 

Written 
Q.C. 

NO  (Code) 

COMMENT 

•□01094 
•□01097 
•□01100 
•□01103 

•□01095 
•□01098 
•□01101 
•□01104 

*□  01096 

•□01099 
•□01102 
*□  01105 

Methyl  red 

Nitrate 

Oxidase 

Voges-Proskauer 

•□01106 
•□01109 
•□01112 
•□01115 

•□01107 
•□01110 
•□01113 
•□01116 

•□01108 
•□01111 
•□01114 
•□01117 

17.    The  following  disc  and/or  strips  are  tested  Bacitracin 
when  each  new  vial  is  opened  and  each  week  Optochin 
of  use  to  demonstrate  positive  and  ONPG 
negative  biochemical  reactions.  X 

V 
XV 

•□01118 
•□01121 
•□01124 
•□01127 
•□01130 
•□01133 

•□01119 
•□01122 
•□01125 
•□01128 
•□01131 
•□01134 

•□01120 
•□  01123 
•□  01126 
•□01129 
•□01132 
•□01135 

18.    The  following  antisera  are  tested  wnen  each          E.  Coli 

new  vial  is  opened  and  each  nionih  of  use  to  Hemophilus 
demonstrate  positive  and  negative  agglutina-  Herella 
tion  reactions  Salmonella 

Shigella 
Streptococcal 

•□01136 
•□01139 
•□01142 
•□01145 
•□01148 
•□01151 

•□01137 
•□01140 
•□01143 
•□01146 
♦□01149 
•□01152 

•□01138 
•□01141 
•□01144 
•□01147 
♦□01150 
♦□01153 

STAINS  CFR  74  21(a)  405. 1313(b)(  1 )(,) 

19.    The  following  stains  are  tested  each  day  of  c?PSl!!e 
use  to  demonstrate  expected  staining  Hagella 
characteristics.                                        Methylene  Blue 

Spore 

•□01154 
•□01157 
•□01160 
s  □  0 1 1 63 

♦□01156 
♦□01159 
♦□01162 
♦□01165 

20.     Gram  stain  is  tested  when  prepared  and  each  week  of  use  to  demonstrate 
expected  staining  characteristics  (positive  and  negative  reactions) 

•□01166 

•□01167 

♦□01168 

21.     Each  batch  of  the  following  media,  if  used  by  the  lab,  is  tested  for  the  following  characteristics 


GROWTH  MEDIUM 

Support 
Growth 
NO  (Code) 

Sterility 
No  (Code) 

Written 
Q.C. 

NO  (Code) 

COMMENT 

Blood  Agar 

Brain  Heart  Infusion  Agar/Broth 
Chocolate  Agar 
Heart  Infusion  Agar 
Motility 

♦□  01169 
♦□  01172 
♦□  01175 
♦□  01178 
•□  01181 

♦□  01170 
♦□  01173 
♦□  01176 
*□  01179 
♦□  01182 

♦□  01171 
♦□  01174 
♦□  01177 
♦□  01180 
♦□  01183 

Mueller  Hinton 
Nutrient  Agar/Broth 
Thioglycollate  Broth 
Tryptic  Soy  Agar/Broth 
Todd  Hewitt  Broth 

•□  01184 
•□  01187 
♦□  01190 
♦□  01193 
*□  01244 

♦□  01185 
♦□  01188 
♦□  01191 
♦□  01194 
♦□  01246 

♦□  01186 
•□  01189 
♦□  01192 
♦□  01195 
♦□  01247 

CDC  3.1005  8-77  BACTERIOLOGY  —  Page  2  of  4  Pages 


ATTACHMENT  1.     EXAMPLE  OF  ONE  PAGE  OF  A  CHECKLIST  USED  BY 
CDC  EXAMINER  DURING  ON-SITE  LABORATORY  EVALUATION 


re 


FIGURE  1.     PROFICIENCY  TESTS  SAMPLES  PREPARED  AND  DISTRIBUTED  BY  CDC 

ll 


NUMBER  OF  LABORATORIES  IN  CDC  PROFICIENCY 
TESTING  PROGRAM,  BY  CATEGORY,  1978 


CATEGORY 

NUMBER  OF 
LABORATORIES 

MICROBIOLOGY 

1,229 

Bacteriology 

872 

Special  Gonorrhoeae  Testing 

353 

Mycobacteriology 

410 

Mycology 

438 

Parasitology 

676 

Virology 

107 

Antimicrobial  Susceptibility  Test 

872 

DIAGNOSTIC  IMMUNOLOGY 

986 

Immunology 

771 

Hepatitis  B  Antigen 

371 

Syphilis  Serology 

552 

CHEMISTRY 

1,555 

Chemistry  Profile 

1,079 

Endocrinology 

708 

Pharmacology- Digox  in 

523 

Toxicology-Drugs  of  Abuse 

455 

Toxicology-Blood  Lead 

238 

HEMATOLOGY 

1,187 

IMMUNOHEMATOLOGY 

700 

CYTOPATHOLOGY 

475 

i 
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DEGREE  OF  IMPLEMENTATION  OF  PREVENTIVE  MAINTENANCE  PROGRAMS 
IN  172  CLIA  LICENSED  LABORATORIES  SINCE  PROGRAM  INCEPTION 
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COMPOSITE  OF  STANDARDS:  Preventive  maintenance  program 


O  1234567 
(69)     (70)      (71)     (72)     (73)     (74)     (75)  (76) 
Years  after  Initial  Examination 


FIGURE  2 


DEGREE  OF  ADOPTION  OF  CLIA  STANDARD  IN  BACTERIOLOGY 
AMONG  37  LABORATORIES  IN  PROGRAM  FROM  ITS  INCEPTION 

lOOi 


43.25  43.25 


64.87 


54.06 


67.57 


72.98 


83.79 


83.79 


STANDARD:  Each  batch  of  media  checked  for  sterility,  ability  to 
support  growth,  and  proper  reaction 


O  1234567 
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Years  after  Initial  Examination 


FIGURE  3 


DEGREE  OF  ADOPTION  OF  CLIA  STANDARD  IN  BACTERIOLOGY 
AMONG  19  LABORATORIES  ENTERING  PROGRAM  IN  1974 
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STANDARD:  Each  batch  of  media  checked  for  sterility,  ability  to 
support  growth,  and  proper  reaction  
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FIGURE  4 


DEGREE  OF  ADOPTION  OF  QUALITY  CONTROL  STANDARDS  BY  CLASS 
OF  1971  LABORATORIES  AND  BY  CLASS  OF  1977  LABORATORIES 


Category 


Percent  of  Class  of  1971  labs 
met  standards  on  date  shown 


Percent  of 
Class  of  1977 


and  standard 

'71 

'72 

'73 

'74 

'75 

'76 

'77  met 

standards 

Bacteriology 

Reagents  checked 

14 

18 

27 

45 

64 

70 

82 

43 

Autoclave  checked 

43 

64 

91 

82 

100 

90 

100 

100 

indate  reagents  used 

29 

55 

64 

91 

82 

100 

100 

100 

Serology 

Tests  controlled 

33 

60 

64 

64 

70 

90 

91 

57 

Chemistry 

Controls,  pregnancy  test 

38 

83 

100 

92 

91 

91 

92 

100 

Immunohematology 

Serum,  cells  checked 

50 

45 

82 

73 

100 

100 

100 

83 
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Percent  of  Laboratories  Making  Grades  of  Less  Than  75, 


BACTERIOLOGY 

NO.  OF  LABS:  (149-736) 
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FIGURE  5 


SUCCESS  OF  CLASS  OF  1971   LABORATORIES  COMPARED  TO  CLASS  OF 
1977  LABORATORIES  IN  ISOLATING  AND  IDENTIFYING 

NEISSERIA  GONORRHOEAE  FROM  MIXED  CULTURES 


Performance  of 


Laboratory 

Performance 

of 

1971   labs  on 

date  shown 

Lab 

1977  labs  in 

Number 

'72 

'73 

'75 

'76 

'76 

'77 

'78 

Number 

'78 

04-60 

NE* 

S 

S 

S 

S 

S 

S 

01-26 

S 

12-31 

NE 

U 

S 

NP 

s 

S 

S 

04-40 

NE 

13-13 

NE 

S 

s 

S 

s 

s 

S 

04-70 

U 

14-05 

NE 

u 

s 

S 

s 

s 

u 

10-30 

S 

20-36 

S 

s 

s 

S 

s 

s 

s 

14-25 

S 

22-14 

u 

u 

s 

s 

s 

s 

s 

15-24 

S 

29-07 

u 

u 

s 

s 

s 

s 

s 

19-68 

S 

32-12 

NP 

NP 

s 

S 

s 

s 

s 

26-16 

S 

40-04 

s 

s 

s 

s 

s 

s 

s 

29-66 

NE 

41-17 

s 

u 

u 

s 

s 

s 

s 

35-07 

U 

45-16 

u 

u 

s 

s 

s 

s 

s 

35-21 

S 

36-40 

S 

37-94 

S 

Percent  S 

43 

36 

91 

91 

100 

100 

~9T 

82 

*NE,  not  enrolled  for  gonorrhea;  S,  satisfactory;  U,  unsatisfactory;  NP,  did  not 
return  results 
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PERFORMANCE  OF  LABORATORIES  RELATED  TO 
DAILY  TESTING  LOAD- NEISSERIA  GONORRHOEAE 


Survey  1976  1      19771        1977  H  19781 

No.  Test  Samples  2  1  3  2 

Percent  of  Laboratories  Correctly  Reporting 
N.  gonorrhoeae,  By  Shipment 


FIGURE  6 


SENSITIVITY  OF  SECOND  AND  THIRD  GENERATION 
TESTS  FOR  DETECTING  HEPATITIS  B  SURFACE 


ANTIGEN 

POSITIVE 

SERA 

Year 

Results 

on  Proficiency  Testing  Samples 

2nd  generation  tests 

3rd  generation  tests 

#  Labs  % 

Correct 

#  Labs 

%  Correct 

1975 

114 

79 

206 

99 

1976 

90 

60 

238 

99 

53 

62 

251 

95 

1977 

48 

66 

274 

98 

48 

80 

273 

99 

1978 

27 

97 

281 

99 

33 

79 

276 

96 

1979 

5 

60 

322 

99 

Average  Percent  Correct 

72 

98 

TABLE  4 
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DISTRIBUTION  OF  RHEUMATOID  FACTOR 
TEST  RESULTS 


CO 
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cr 
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40 
20 
0 


Nonstandardized 
Variance  =4.21 


t128^64  f32  -16  *8  -4  t2       x2  x4  x8  xl6 


Actual 
Standardized  Values 
Variance  =  0.93 


-4  ^2       x2  x4  x8 


Dilutions  Expressed  as  Multiples  or  Fractions 
of  The  Median  Value 


figure  7 


Percent  of  Laboratories  with  Results  Outside 
Limits  of  ±  .5  x  10l2/l  around  Target  Value 
Red  Blood  Cell  Count 

NO.  OF  LABS :( 129- 640) 
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FIGURE  8 
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National  Bureau  of  Standards  Special  Publication  591,  Proceedings  of  the  National  Conference 
pn  Testing  Laboratory  Performance:  Evaluation  and  Accreditation,  held  at  NBS ,  Gaithersburg, 
[fl),  September  25-26,  1979.        Issued  August  1980 


EVALUATION  AND  APPROVAL  OF  LABORATORIES  IN  CONNECTICUT 
IMPORTANCE  OF  VOLUNTARY  STANDARDS 

JESSE  S.  TUCKER,  PH.D. 

LABORATORY  DIVISION 
CONNECTICUT  DEPARTMENT  OF  HEALTH  SERVICES 
HARTFORD,  CONNECTICUT  06101 


The  Laboratory  Division  of  the  Connecticut  Department  of  Health 
Services  is  responsible  for  licensure,  registration,  performance  eval- 
uation and  approval  of  hospital,  indpendent  clinical,  water,  waste 
water,  food,  dairy,  air  and  a  variety  of  other  laboratories  which  per- 
form tests  of  public  health  significance.     Of  the  almost  400  facilities, 
more  than  three-quarters  of  these  participate  in  proficiency  testing 
and  are  inspected  yearly. 

The  philosophy  of  performance    evaluation  and  approval  in  Connecti- 
cut has  evolved  from  one  of  complete  authority  and  control  by  the 
state  to  one  which  is  based  increasingly  upon  cost  effectiveness,  con- 
sistency, enforceability  and  participant  input.     Dependence  on  voluntary 
standards  now  extends  from  granting  equivalence  for  inspections  conducted 
by  qualified  professional  groups  to  accepting  proficiency  testing  re- 
sults from  well  conceived  programs.     Most  success  in  acceptance  of 
equivalent  programs  has  been  attained  with  clinical  laboratories  and 
least  progress  has  been  made  with  environmental  laboratories.  This 
difference  probably  is  attributable  to  the  newness  of  environmental  as 
opposed  to  clinical  laboratory  legislation  and  the  availability  of  accept- 
able programs. 

Using  the  evaluation  and  approval  of  recombinant  DNA  laboratories 
in  Connecticut  as  an  example,  the  advantages  of  using  existing  regula- 
tions and  voluntary  standards  are  discussed  as  a  model  of  efficiency. 

Key  words:     Clinical  laboratories;  cost  effectiveness;  environmental 
laboratories;  licensure;  performance  evaluation;  physician's  office 
laboratories;  proficiency  testing;  public  health;  recombinant  DNA; 
registration  and  approval;  voluntary  standards. 


1.  Introduction 

Responsibilities  of  State  Agencies  for 
regulation  of  laboratories  are  increasing 
more  rapidly  than  the  funds  available  to  ful- 
fill these  responsibilities.     An  innovative 
approach  is  needed  to  assure  quality  of  lab- 
oratory results  without  excess  expenditure 
of  taxpayers  funds. 

After  almost  ten  years  of  a  traditional 
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regulatory  program  in  which  state  resources 
were  required  to  maintain  complete  responsi- 
bility for  inspection  and  proficiency  testing 
programs,  the  Connecticut  Health  Laboratory 
has  initiated  a  system  involving  voluntary 
standards  and  participation  by  non-govern- 
mental organization  in  the  inspection 
and  proficiency  testing  process.  Specifics 
of  the  Connecticut  program  including  ad- 
vantages, limitations  and  the  need  for 
continued  review  and  change  are  covered  in 
this  report. 


2.  Scope 

In  Connecticut,  the  Laboratory  Division 
of  the  Department  of  Health  Services  is  res- 
ponsible for  licensure,  registration,  in- 
spection, proficiency  testing,  and  approval 
of  all  laboratories  performing  tests  of 
public  health  significance  on  or  for  Connect- 
icut citizens.     Included  are  hospital  and 
independent  laboratories,  water,  waste 
water,  diary,  food  and  air  testing  labora- 
tories, blood  banks  and  plasmapheresis 
centers.     Recombinant  DNA  laboratories, 
academic  laboratories  and  laboratories  using 
dogs  for  teaching  and  research  are  also  in- 
cluded.    The  Department  of  Health  Services 
has  primary  enforcement  responsibility  for 
approval  of  laboratories  included  under  Medi- 
care, the  Clinical  Laboratories  Improvement 
Act  (CLIA),  the  Safe  Drinking  Water  Act 
(SDWA)  and  National  Polution  Discharge  Eli- 
mination Systems  (NPDES)  legislation.  About 
400  separate  facilities,  as  well  as  the 
proficiency  testing  specimens  required  to 
determine  quality  of  the  work  done  by  them, 
are  the  responsibility  of  ten  professional 
and  four  non-professional  employees.  These 
regulatory  responsibilities  cannot  be 
discharged  effectively  without  at  least 
tacit  acceptance  of  the  basic  principles  of 
voluntary  standards. 

3.  Clinical  Laboratory  Regulation 

At  the  request  of  the  Connecticut  So- 
ciety of  Pathologists  and  members  of  the 
State  Advisory  Committee  on  Clinical  Labor- 
atory Regulation  the  Laboratory  Division 
began  looking  at  alternatives  to  state 
sponsored  inspection  and  proficiency  testing 
programs.     Hospital  and  independent  clinical 
laboratories  had  long  been  subjected  to  a 
multiplicity  of  programs  required  by  federal, 
state  and  peer  review  groups.     After  care- 
fully reviewing  the  inspection  and  accredita- 
tion program  of  the  American  Association  of 
Blood  Banks  (AABB)  an  agreement  for  alternate 
year  inspections  by  the  state  and  AABB  was 
concluded  in  November  of  1977.     The  most 
significant  reasons  for  this  decision  were: 

1.  The  AABB  program  was  more  thorough 
than  ours. 

2.  Funding  was  not  available  to  provide 
an  equivalent  program. 

A  somewhat  different  agreement  was 
reached  with  the  College  of  American  Patholo- 
gists (CAP)  in  December  of  1978.  In  essence, 
the  State  of  Connecticut  accepted  the  bi- 
ennial inspection  of  member  hospital  lab- 
oratories in  lieu  of  the  yearly  state  in- 
spection.    However,  CAP  agreed  to  a  10% 
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reinspection  by  the  state.     About  one  half 
of  the  50  hospitals  are  accredited  by  CAP 
leaving  the  state  with  about  twenty  five 
annual  surveys  and  reinspections .     It  is 
expected  that  some  agreement  soon  will  be 
worked  out  with  the  Joint  Commission  on  th( 
Accreditation  of  Hospitals  (JCAH)  because  i 
the  recent  "deemed  status"  granted  to  this 
organization  by  the  federal  government  to 
approve  hospital  laboratories  for  Medicare 
purposes.     It  was  not  altruism  which  prompt 
this  arrangement .     Rather ,  it  was  the  in- 
creasing realization  that  federal  funding 
would  not  always  be  available,  that  regula-I 
tion  is  not  one  of  the  more  popular  items 
competing  for  state  funds,  that  these  inst: 
tutions  are  providing  quality  laboratory 
services  as  evidenced  by  eight  years  of 
acceptable  proficiency  testing  results  and 
that  redundancy  in  inspection  was  creating 
more  problems  than  it  was  solving. 

Much  of  the  impetus  for  statutory  chai 
(which  in  January  1,1980  will  enable  accepl 
ance  of  proficiency  testing  programs  other 
than  those  provided  by  the  state)  originate 
from  the  sub-committee  for  Microbiology  whi 
functions  as  part  of  our  mandated  Advisory 
Committee  for  Clinical  Laboratory  regulatic 
This  advisory  committee  includes  peer  revic 
and  has  provided  a  perspective  unavailable 
to  many  regulatory  agencies.  Input  from  hos 
tal  and  independent  laboratory  microbiolo- 
gists has  resulted  in  an  appreciation  of  tl 
differences  in  problems  encountered  in  dif -j- 
ferent     settings  as  well  as  common  problems 
resulting  from  inspection  and  proficiency 
testing.     The  disadvantages  usually  expects 
from  this  "fox  in  the  chicken  coop"  approac 
have  not  materialized. 


Another  problem  area  which  was  improve" 
by  a  change  in  statute  was  licensure  of 
physician's  office  laboratories.  Previousl 
a  combined  practice  of  two  or  more  practi- 
tioners  required  licensure.     Only  a  single 
physician  who  performed  tests  in  the  course 
of  his  practice  was  exempted.     This  law  was 
unreasonable  and  unenforceable.     The  new 
law  establishes  a  licensure  requirement  for 
practitioners  which  is  based  on  volume  and  1 
complexity  of  testing  as  defined  in  regula- 
tions.    We  feel  that  this  is  a  more  logical 
and  economical  approach  than  requiring 
licensure  for  all  practitioners  performing 
tests  or  basing  an  exemption  on  the  number 
of  practitioners  involved  such  as  the 
arbitrary  five  limit  suggested  by  a  recent  1 
version  of  the  proposed  CLIA. 

Peer  review  is  not  without  problems. 
Most  peer  reviewers  are  not  thorough  enough 
in  covering  details.     This  thoroughness  is 


criticized  frequently  as  nit-picking. 
Any  professional  has  available  to 
hiai  textbooks,  journals,  seminars  and 
continuing  education  courses  to  satisfy  his 

i(    intellectual  requirements.     However,  he  does 
not  always  have  and  needs  periodic  scrutiny 

■  to  make  certain  that  he  and  his  employees 
are  properly  taking  and  labeling  specimens, 

5  updating  specifications,  recording  informa- 
tion necessary  to  achieve  adequate  quality 
control,  protecting  themselves  from  biologi- 
cal and  chemical  danger  and  accurately  re- 

j  porting  results.     Some  of  the  most  astute 
laboratory  scientists  are  remiss  at  keeping 
records  of  their  own  work  and  helpless  at 
requiring  others  to  do  so.     There  is  no 

i  better  way  to  be  assured  that  records  will'  be 
kept  properly  than  to  convince  the  laboratory 
director  or  supervisor  that  his  errors  and 
omissions  will  be  discovered. 

Additional  problems  in  peer  review  are 
created  by  differences  in  the  frequency  of 
inspections.     An  unfair  burden  is  placed  on 
independent  laboratories  which  are  required 
to  be  inspected  yearly  under  the  same  Medi- 
care regulations  which  allow  on-site  inspect- 
ion by  JCAH  and  CAP,  biennially.  Inequities 
also  exist  in  the  criteria  used  during  in- 
spection.    A  classic  example  with  which  our 
group  has  been  wrestling  for  some  time  is  the 
Medicare  requirement  for  use  of  quality 
control  organisms  on  a  daily  basis  for  anti- 
biotic susceptibility  testing  CI]  which  is 

less  stringent  for  CAP  approved  labora- 
tories  121  . 

Application  of  the  term  "voluntary"  to 
proficiency  testing  and  laboratory  inspection 
has  resulted  in  some  differences  in  inter- 
pretation.    The  pathologist  who  participates 
in  the  CAP  proficiency  testing  program  may 
make  the  interpretation  that  he  participates 
in  the  program  of  his  own  volition,  that  he 
can  withdraw  at  any  time  and  that  there  are 
no  penalties  if  his  performance  is  not  with- 
in the  prescribed  limits.     However,  when  a 
governmental  agency  accepts  such  a  program 
to  meet  requirements  for  regulatory  purposes 
the  only  voluntary  thing  about  it  is  that 
he  has  chosen  this  route  rather  than  be 
subjected  to  the  alternative  state  program. 
It  is  incumbent  upon  the  agency  granting 
equivalent  status  to  differentiate  between 
the  good  and  poor  performer  and  to  use  this 
information  to  require  improvement  or  to 
prevent  any  damage  from  poor  results.  Input 
and  compliance  are  the  keys  to  voluntary 
standards.     Particpants  must  have  the 
opportunity  not  only  to  criticize  the  stan- 
dards currently  in  use  and  to  make  suggest- 
ions for  change  but  also  to  make  compromises 
and  to  adhere  to  the  standards  once  adopted. 


A  cost  effective  program  for  determining 
quality  of  laboratory  performance  will  be 
forthcoming  when  proficiency  tests  can  be 
conducted  without  prior  warning  and  when  the 
correct  response  has  been  established  well 
in  advance  of  the  actual  test.     Several  years 
ago  when  this  suggestion  was  made  to  Center 
for  Disease  Control  by  our  group  it  was  con- 
sidered to  be  impractical.     However,  in  the 
recent  Public  Health  Service  publication 
"Proposed  Interim  Proficiency  Testing  Pro- 
gram for  Medicare  Laboratories"   [3]  methods 
for  reference  laboratories  were  recommended 
for  the  chemistry  program.     It  is  expected 
that  additional  definitive  and  reference 
methods  [4]  will  be  established  by  groups 
such  as  The  National  Committee  for  Clinical 
Laboratory  Standards  (NCCLS) . 

At  the  Connecticut  State  Health  Lab- 
oratory it  is  our  opinion  that  a  better 
appraisal  of  laboratory  performance  in 
clinical  chemistry  can  be  obtained  by  two 
unannounced  proficiency  tests  than  by  the 
six  announced  tests  currently  recommended  in 
the  above  reference  by  the  Public  Health 
Service.     Three  specimens,  two  unknowns 
(normal  and  abnormal)  and  one  specimen  with 
known  values  would  be  used.     All  specimens 
would  be  assayed  using  reference  methods. 
Statistically  significant  target  values  and 
acceptable  ranges  would  be  established  in 
advance.     After  a  suitable  period  to  allow 
all  laboratories  to  adjust  their  procedures 
to  obtain  values  comparable  to  those  from 
the  reference  methods,  the  specimen  with 
assigned  values  would  be  discontinued.  A 
similar  approach  could  be  used  for  categories 
other  than  chemistry. 

After  all  the  time  and  effort  that  has 
been  directed  to  performance  evaluation 
through  proficiency  testing,  a  serious  pro- 
blem still  remains.     Little  progress  has 
been  made  in  development  of  a  system  to 
determine  integrity  of  the  specimens.  Until 
such  a  system  is  developed,  no  assessment 
can  be  made  of  the  significance  of  specimen 
handling  (including  methods  for  obtaining, 
processing  and  transporting  the  specimen)  on 
quality  of  laboratory  performance. 

4.     Environmental  and  Public  Health 
Laboratories 

The  most  complex  regulatory  problems 
now  exist  in  the  environmental  area.  Some 
of  the  precise  methodological  requirements 
of  the  Safe  Drinking  Water  Act  and  the  Water 
Pollution  Control  Act  are  difficult  to  com- 
prehend.    Good  proficiency  test  specimens 
for  volatiles  and  chlorinated  hydrocarbons 
are  hard  to  find  and  we  now  prepare  them  in- 
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house.     We  have  not  yet  had  the  opportunity  to 
to  consider  the  problems  involved  in  deter- 
mining quality  of  air  analysis.     Even  the 
variety  of  analyses  performed  in  different 
laboratories  is  surprising.     For  example, 
there  are  19  laboratories  in  Connecticut 
approved  to  perform  tests  required  under  the 
Safe  Drinking  Water  Act,  ten  approved  under 
NPDES  and  48  approved  for  both.     Of  the  48 
having  combined  approval,  8  of  these  are 
approved  to  do  food  and  one  of  these  also 
performs  tests  for  pollutants  in  air.  Four 
laboratories  are  approved  to  perform  tests 
only  on  domestic  waste  water.     Eight  labora- 
tories in  Connecticut  are  approved  to  perform 
tests  on  milk  as  required  by  State  and  Inter- 
state Milk  Shippers  regulations.     Most  of 
the  requirements  for  inspection  and  profi- 
ciency testing  of  laboratories  in  these 
specialties  currently  are  prescribed  with 
very  little  leeway  for  application  of  vol- 
untary standards. 

5.     Recombinant  DNA  Laboratories 

Laboratories  which  lend  themselves  best 
to  voluntary  standards  are  the  academic, 
quality  control  and  Recombinant  DNA  Labora- 
tories.    They  require  registration  because  of 
their  use  of  organisms  of  public  health  sign- 
ificance.    Recombinant  DNA  laboratories  fit 
into  this  category  because  E.  coli  is  in- 
cluded in  the  public  health  code  as  one  such 
organism.     They  are  registered  under  exist- 
ing regulations  because  those  involved 
finally  agreed  that  this  was  a  lesser  evil 
than  a  whole  set  of  new  state  statutes  and 
regulations  which  could  differ  substantially 
from  existing  federal  requirements.  The 
only  state  requirement  is  that  these  facili- 
ties adhere  strictly  to  NIH  guidelines, 
whether  or  not  their  work  is  federally 
supported,  and  that  they  keep  the  state  in- 
formed of  changes  in  projects  and  advisory 
committee  members.     They  have  been  inspected 
yearly  and  found  not  only  to  obey  the  guide- 
lines but  also  to  be  an  excellent  source  of 
information  on  laboratory  safety.     This  model 
of  simplicity  in  laboratory  regulation  is  also 
a  model  for  the  application  of  voluntary 
standards  in  an  area  which  the  public  con- 
siders to  be  critical  for  protection  of  their 
health  and  safety. 


1  am  grateful  to  John  Redys,  Director  of  the 
Laboratory,  and  members  of  the  regulatory 
staff  for  critically  reading  the  manuscript 
and  to  Phyllis  Botticello  for  assisting  in  its 
preparation. 
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THE  LABORATORY  ACCREDITATION  PROGRAM  OF  THE  AMERICAN  INDUSTRIAL  HYGIENE  ASSOCIATION 
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SENIOR  ENVIRONMENTAL  HEALTH  CHEMIST,  BETHLEHEM  STEEL  &  CHAIRMAN 
AMERICAN  INDUSTRIAL  HYGIENE  ASSOCIATION  LABORATORY  ACCREDITATION  COMMITTEE 


The  accreditation  program  for  laboratories  will  be  described  along 
with  the  program  to  assist  accredited  laboratories  to  maintain  their 
accreditation  through  continued  demonstration  of  competence  and  perfor- 
mance.    An  Industrial  Hygiene  Laboratory  is  a  laboratory  that  analyzes 
samples  taken  to  assist  in  the  recognition,  evaluation  and  control  of 
the  workplace  environment.     The  factors  which  are  used  to  determine 
whether  a  laboratory  qualifies  for  accreditation  as  an  industrial 
hygiene  laboratory  will  be  described  in  detail.     Such  factors  include 
personnel  qualifications,  quality  control  and  proficiency  testing, 
facilities,  recordkeeping,  analyses  performed,  safety  and  analytical 
procedures.     A  description  of  the  accreditation  process  will  be  pro- 
vided along  with  steps  that  are  taken  to  assist  laboratories  not  main- 
taining acceptable  standards  of  performance.     The  role  of  the  coordi- 
nator, the  site  visitor,  the  accreditation  committee  and  the  AIHA 
Board  of  Directors  will  be  outlined  and  possible  changes  in  the  pro- 
gram wi] 1  be  discussed.     The  effectiveness  of  the  program  in  improving 
industrial  hygiene  laboratories  will  be  reviewed. 

Key  words:     Accreditation;  Accreditation  Committee;  American  Industrial 
Hygiene  Association;  coordinator;   industrial  hygiene  laboratories; 
laboratory;  PAT;  performance;  personnel;  proficient;  quality  control; 
reaccreditation;  site  visit 


The  first  step  in  discussing  the  lab- 
oratory accreditation  program  of  the 
American  Industrial  Hygiene  Association  is 
to  define  the  term  industrial  hygiene  and 
briefly  describe  an  industrial  hygiene  lab- 
oratory.    Industrial  hygiene  is  that  science 
and  art  devoted  to  the  recognition,  evalua- 
tion and  control  of  those  environmental 
factors  or  stresses,  arising  in  or  from  the 
workplace,  which  may  cause  sickness,  im- 
paired health  and  well  being,  or  signifi- 
cant discomfort  and  inefficiency  among  work- 
place employees. 

The  industrial  hygienist  depends  on  the 
industrial  hygiene  laboratory,  a  laboratory 
that  analyzes  samples  taken  to  assist  in  the 
recognition,  evaluation  and  control  of  the 
workplace  environment  for  data.  The 


laboratory  is  critical  for  determining 
existing  potential  health  hazards  and 
measuring  the  effectiveness  of  control 
measures  implemented  in  previously  identi- 
fied problem  areas.     The  laboratory  data  is 
interpreted  by  the  industrial  hygienist  who 
must  then  relate  the  findings  to  standards 
set  by  the  Occupational  Safety  and  Health 
Administration  (OSHA)  -  with  advice  from  the 
National  Institute  for  Occupational  Safety 
and  Health  (NIOSH) .     It  is  important,  there- 
fore,  that  the  laboratory  provide  data  that 
can  be  trusted. 

An  examination  of  these  definitions 
shows  a  definite  need  for  some  method  of 
insuring  the  validity  of  the  data  such  as 
accreditation  of  the  laboratories. 
Accreditation  means  professionalism  to  those 
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laboratories  involved  in  the  program.     It  is 
a  mark  of  competence  demonstrating  that  the 
laboratory  measures  up  to  the  strictest 
possible  standards.     The  sole  aim  of  the 
American  Industrial  Hygiene  Association 
accreditation  program  is  to  assist  industrial 
hygiene  laboratories  in  achieving  and 
maintaining  the  highest  possible  level  of 
professional  performance. 

If  a  laboratory  is  not  accredited, 
there  is  no  assurance  that  the  data  or 
information  that  it  generates  is  reliable. 
The  accreditation  program  has  helped  lab- 
oratories find  areas  where  improvement  was 
needed  and  has  assisted  laboratories  in 
development  of  quality  control  programs 
that  insure  valid  data.     All  laboratories 
participating  in  the  accreditation  program 
will  probably  agree  that  the  program  has 
been  beneficial.     Accreditation  is  a  state- 
ment to  both  the  public  and  to  workplace 
employees  who  are  affected  by  the  results 
that  a  laboratory  has  met  certain  standards 
set  by  recognized  professionals  in  the  field. 
A  laboratory  accreditation  program  is  and 
will  continue  to  be  an  effective  tool  in 
the  continued  improvement  in  the  perfor- 
mance of  the  industrial  hygiene  laboratory. 


How  does  the  accreditation  process 


work? 
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The  flow  chart  in  Figure  1  shows  the  variot 
steps  in  the  process  leading  to  accredita- 
tion along  with  those  places  where  assist- 
ance can  be  provided.     First  is  the  reques 
for  the  application  which  is  filled  out  am 
sent  to  the  laboratory  coordinator  at  the  ;£ 
American  Industrial  Hygiene  headquarters  ii 
Akron,  Ohio.     The  coordinator  who  is 
employed  by  the  Association  reviews  the 
application,  mainly  for  completeness  and 
then  sends  the  application  to  two  members 
of  the  Laboratory  Accreditation  Committee 
for  a  thorough  review.     The  Laboratory 
Accreditation  Committee  is  a  committee  of  | 
qualified  volunteers  who  are  appointed 
annually  by  the  AIHA  Board  of  Directors 
The  laboratory  is  simultaneously  enrolled 
in  the  NIOSH  Proficiency  Analytical  Test- 
ing program,  better  known  as  PAT.  This 
program  requires  bi-monthly  evaluation  of 
supplied  contaminants.     Satisfactory  per- 
formance judged  against  the  results  of  all 
participating  laboratories  is  required  to  (ftk 
become  accredited  and  to  remain  accredited 


The  first  aspect  of  the  application 
which  is  considered  important  are  the 
qualifications  of  personnel.     The  director 
of  the  laboratory  must  be  a  Diplomate  of 
the  American  Board  of  Industrial  Hygiene  ot 
have  six  years  experience  in  industrial 
hygiene  chemistry  after  completing  a  medics 
degree  or  an  academic  degree  in  one  of  the 
basic  sciences.     The  committee  must  judge 
whether  or  not  the  applicant  meets  the  six 
years  experience  standard.     The  entire  work 
experience  of  the  individual  is  considered  jj 
to  make  certain  that  the  laboratory  is  unde: 
the  control  of  trained,  competent,  practic-j 
ing  industrial  hygiene  chemists. 

This  requirement  is  noteworthy  for  the 
following  reasons.     Historically  there  has 
been  good  corelation  between  the  ease  with 
which  a  new  laboratory  has  met  this  ex- 
perience criteria  and  measurements  of  per- 
formance subsequently  achieved  and  main- 
tained.    The  experience  requirement  has 
often  been  a  big  hurdle  for  many  labora- 
tories -  particularly  those  where  the 
industrial  hygiene  function  has  been  a  non-: 
specific  entity  of  a  larger  general 
chemistry  laboratory. 

The  laboratory  supervisor  should  have  a; 
doctorate  degree  in  a  related  science  or  a 
medical  degree  with  an  additional  two  years 
experience  in  related  procedures.  Other- 
wise, he  should  have  a  master's  degree  plus 
four  years  experience  or  a  baccalaureate 
degree  and  six  years  experience  in  related 
procedures.     There  are  also  requirements 
spelled  out  for  the  technologists  and 
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j:  technicians  .     The  titles  often  do  not  match 
•those  used  internally  within  the  various 
•organizations  seeking  accreditation,  how- 
i: ever,  the  program  is  more  concerned  with  the 

function  of  the  individuals  rather  than  with 

the  title. 

The  area  of  personnel  qualifications 
most  often  raises  questions  such  that  the 
laboratory  may  have  to  provide  additional 
detailed  information  prior  to  review.  The 
subcommittee  review  may  also  suggest  that 
someone  else  may  be  more  appropriate  as 
director  until  specific  requirements  are 
attained  by  the  named  individual.     At  the 
'time  of  the  site  visit,  suggestions  again 

fmay  be  made  with  regard  to  improving 
qualifications  and  helping  the  individuals 

'to  become  more  familiar  with  industrial 
health.     Suggestions  which  are  made  are  not 
binding  on  management  but  are  offered  so 
that  personnel  structures  will  compare  with 
those  of  other  accredited  laboratories. 

J 

Closely  related  to  the  importance  of 
having  trained  industrial  hygiene  chemists 
jis  the  decision  to  accredit  only  labora- 
tories performing  a  significant  amount  of 
industrial  hygiene  work.     This  sounds 
straightforward  but  is  difficult  to 
administer  in  practice.     Many  laboratories 
are  involved  with  water  chemistry,  air 
pollution,  solid  waste  or  toxicology. 
There  are  also  laboratories  that  want  to  be 
accredited  under  the  program  that  do  not 
i  have  any  identifiable  industrial  hygiene 
I  program.     To  measure  the  involvement  of  a 
i  laboratory  in  industrial  hygiene  the 
application  requires  a  statement  of  work 
load  in  terms  of  origin  and  purpose 
encompassing  the  past  year's  testing. 

A  second  area  of  importance  is  in  re- 
gard to  the  laboratory  quality  control  pro- 
gram.    In  committee  review,  the  quality 
control  program  is  given  equal  weight  with 

I  the  qualification  of  the  personnel.     If  a 
laboratory  does  not  have  a  good  quality 
control  program,   it  will  be  very  difficult 
for  any  laboratory  to  obtain  accreditation. 
The  requirements  include  a  statement  of 
policy  with  regard  to  quality  control  and  a 
manual  which  contains  all  quality  control 
work.     Preferably,  someone  should  be 
designated  as  the  laboratory  quality  control 
coordinator,  although  for  smaller  labora- 
tories this  individual  may  serve  in  a 
multiple  capacity.     Control  charts  should  be 

i  in  use  for  those  analyses  most  commonly 
encountered  and  spiked  samples  should  be 
used  for  infrequently  performed  analyses. 
The  quality  control  policy  should  state 
what  action  is  taken  when  a  result  is  found 


outside  the  expected  range.  Participation 
in  other  proficiency  testing  programs  and 
exchange  of  samples  with  other  accredited 
industrial  hygiene  laboratories  is  encourag- 
ed. 

The  application  also  requires  a  detailed 
description  of  other  factors: 

1.  Facilities 

2.  Record  keeping  system 

3.  Type  and  amount  of  analysis  per- 
formed 

4.  Safety  program 

5.  Analytical  procedures 

The  act  of  answering  the  queries  is, 
itself,  beneficial.     Many  interested  labora- 
tories,  in  an  effort  to  respond,  have  recog- 
nized certain  benefical  changes  which  sub- 
sequently have  been  instituted.     In  a  few 
instances,  applications  have  been  delayed 
until  changes  have  been  made.     As  stated 
before,   the  purpose  of  the  program  is  to 
assist  laboratories  in  achieving  and 
maintaining  a  high  level  of  professional 
performance . 

After  an  application  has  been  reviewed 
and  accepted  by  the  subcommittee,   and  a 
laboratory  has  completed  two  PAT  rounds 
successfully,  an  on-site  visit  is  scheduled 
to  verify  the  material  presented  and  to 
examine  the  operation  of  the  laboratory. 
The  site  visitor  is  an  expert  in  the  field 
of  industrial  hygiene  and  his  objective  is 
simply  to  offer  constructive  advice. 

The  site  visitation  is  an  important 
aspect  of  the  accreditation  process.  Site 
visitors  are  selected  based  on  their  exper- 
tise in  laboratory  operations.     Site  visits, 
which  are  scheduled  at  mutually  convenient 
dates,   are  usually  conducted  and  completed 
in  one  day.     If  deficiencies  are  noted, 
recommendations  are  made  to  the  laboratory 
regarding  changes  which  should  be  made. 

Information  from  the  application  review, 
site  visit  report  and  performance  in  the 
PAT  program  are  submitted  for  final  commit- 
tee approval.     The  committee  may  request 
additional  information  or  make  further 
recommendations  to  the  applicant.  In 
some  instances,  a  second  site  visit  may  be 
required  to  insure  that  major  recommenda- 
tions have  been  carried  out.     After  approval 
by  the  committee,   summary  sheets  are  sub- 
mitted for  approval  by  the  Board  of 
Directors  of  the  American  Industrial  Hygiene 
Association . 

Accreditation  is  granted  for  a  period  of 
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three  years.     During  this  period,   a  labora- 
tory is  required  to  participate  in  the  PAT 
program  and  advise  the  committee  on  an 
annual  basis  of  any  major  changes  that  occur. 
A  current  list  of  accredited  laboratories 
appears  in  the  Journal  of  the  American 
Industrial  Hygiene  Association. 

The  reaccreditation  process  operates  in 
a  similar  fashion  except  for  a  certain 
simplification  of  the  procedure.     The  re- 
accreditation  application  basically 
requests  information  on  the  changes  and  im- 
provements that  have  occurred  over  the 
three  year  period.     The  subcommittee  review 
is  not  needed  and  the  site  visit  is  con- 
ducted with  all  the  accumulated  information 
over  the  three  year  period,  including 
previous  site  visitor's  and  committee 
recommendations,  performance  in  the  PAT 
program,   changes  in  personnel  and  any  other 
changes  of  significance  that  may  affect  the 
performance  of  the  laboratory.     The  committee 
then  reviews  the  entire  package  and  passes 
its  recommendation  on  to  the  AIHA  Board  of 
Directors . 

As  has  been  stated  the  purpose  of  the 
program  is  to  assist  laboratories  in  up- 
grading performance.     The  first  place  for 
providing  assistance  is  during  the  co- 
ordinator review  of  the  application.  The 
coordinator's  job  is  to  insure  that  the 
application  is  filled  out  completely  and 
correctly.     He  may  recommend  greater  empha- 
sis in  some  areas  and  less  in  others  not 
only  to  assist  the  laboratory  but  to  help 
the  committee  better  understand  the  nature 
of  a  particular  laboratory  operation.  He 
serves  as  the  communicator,   relaying  re- 
commendations of  both  the  committee  and 
site  visitors  and  notifying  the  laboratory 
when  a  final  decision  is  made. 

The  subcommittee  review  points  out 
areas  where  more  information  may  be  needed 
and  suggests  potential  trouble  spots  that 
the  site  visitor  may  need  to  discuss  with 
the  laboratory. 

The  site  visitor  prepares  a  written 
report  with  recommendations  for  improve- 
ment which  is  part  of  the  total  package  for 
committee  review.     These  recommendations 
must  be  answered  with  specific  statements 
of  action  that  will  be  taken  by  the  labora- 
tory. 

The  results  of  the  PAT  program  are 
used  to  determine  a  laboratory's  pro- 
ficiency for  those  analyses  performed. 
Extra  sets  of  samples  are  available  for 


laboratories  having  difficulties.  Also, 
exchanging  samples  with  other  laboratories 
is  suggested  as  a  help  in  resolving  diffi- 
culties . 

A  meeting  of  laboratory  directors  is 
held  annually  at  which  time  problems  and 
progress  are  discussed.     Also,  a  quarterly 
newsletter  is  published  for  information 
exchange  among  laboratories.     The  newslette 
carries  items  of  interest  submitted  by 
laboratories,  new  products  available,  and 
summary  information  on  PAT  data. 

Although  the  program  has  been  in 
operation  for  over  six  years  changes  are 
under  investigation  to  insure  continued 
improvement.     Most  of  the  laboratories  that 
would  be  classified  as  industrial  hygiene 
laboratories  are  accredited.     The  committee 
must  now  deal  with  what  it  considers  border 
line  laboratories  to  insure  that  these  are 
not  excluded  simply  because  they  are  not 
strictly  industrial  hygiene  laboratories. 
Other  changes  or  items  under  committee 
review  are: 

1.  The  definition  of  proficient  and  non- 
proficient  to  conform  with  the  defini- 
tion used  by  NIOSH. 

2.  The  requirements  for  laboratory  direc- 
tor and  supervisor  which  would  leave 
no  questions  regarding  who  would  or 
would  not  qualify. 

3.  Additional  procedures  for  assisting 
laboratories  that  are  having  diffi- 
culties with  the  PAT  program  to  insure 
proficient  analyses  not  only  for  those 
analytes  in  the  PAT  program  but  for  al~. 
analytes . 

4.  The  possibility  of  an  interim  accredits 
tion  for  those  laboratories  that  may  nc 
exactly  meet  the  requirements  but  whic? 
obviously  have  analytical  capability. 

Response  to  the  accreditation  program 
has  been  excellent.  The  list  of  about  one 
hundred  thirty  (130)  accredited  laboratories 
is  still  growing  with  no  sign  of  decrease  if 
the  rate  of  growth. 

In  summary,  most  laboratories  agree  the 
accreditation  program  has  been  beneficial  tc 
them.     It  is  a  statement  to  both  the  public  ( 
and  employees  that  a  laboratory  has  met 
standards  as  set  by  recognized  professionals 
in  the  field.     The  laboratory  accreditation 
program  is  and  will  continue  to  be  an 
effective  tool  for  the  improvement  in  per- 
formance of  the  occupational  health  labora- 
tory. 
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COLLEGE  OF  AMERICAN  PATHOLOGISTS 
INSPECTION  AND  ACCREDITATION  PROGRAM 

FRANCES  RYAN 

COLLEGE  OF  AMERICAN  PATHOLOGISTS 


The  College  of  American  Pathologists  Inspection  and  Accreditation 
Program  is  a  professional,  voluntary,  peer-review  program  of  laboratory 
improvement.     The  standards  are  developed  by  a  Commission  of  volunteer 
pathologists  and  the  program  is  operated  by  this  Commission,  with  sup- 
port from  the  College's  central  office  and  computer  center.  Biennial 
inspections  are  conducted  by  volunteer  pathologists.     Each  pathologist 
is  assisted  by  a  team  of  his  choosing,  representative  of  the  size  and 
complexity  of  the  laboratory  to  be  inspected.     The  standards  include 
requirements  on  quality  control,  safety,  environment,  instrument  main- 
tenance and  personnel.     An  inspection  checklist  is  used  to  assure  that 
all  areas  are  examined.     Proficiency  testing  is  a  prerequisite  and 
results  are  an  integral  part  of  the  accreditation  decision.  Emphasis 
is  placed  on  the  educational  aspects  of  the  program.     CAP  accreditation 
is  accepted  by  the  Joint  Commission  on  Accreditation  of  Hospitals,  the 
Center  for  Disease  Control,  and  some  states.     Accreditation  by  CAP 
assures  the  physician  and  patient  that  test  results  are  reliable. 

Key  words:     Accreditation;  College  of  American  Pathologists;  criteria; 
inspection;  inspector's  manual;  pathology;  proficiency  testing; 
standards . 


1.  Description 

The  College  of  American  Pathologists 
Inspection  and  Accreditation  Program  was 
developed  almost  20  years  ago  with  the  pri- 
mary objective  of  improving  the  quality  of 
clinical  laboratory  services  and  assuring  the 
accuracy  and  reliability  of  test  results. 
Although  the  program  has  grown  in  complexity 
and  effectiveness,  the  goal  still  remains  one 
of  laboratory  improvement  through  peer-review 
and  education.     The  College  of  American 
Pathologists  Inspection  and  Accreditation 
Program  is  a  professional,  voluntary,  peer- 
review  program  with  about  1800  laboratories 
currently  participating.     This  represents 
more  than  10  percent  of  the  labs  in  the 
country. 


2.     Development  and  Operation 

The  program  is  under  the  direction  of  a 
Commission  on  Inspection  and  Accreditation; 
a  group  comprised  of  a  chairman  and  ten 
regional  commissioners.     Each  commissioner 
is  a  highly  qualified  pathologist  who  has 
been  appointed  by  the  President  of  the 
College.     The  commission  meets  at  least 
quarterly  to  review  its  current  standards 
and  to  examine  new  or  problem  areas  that 
may  exist.     Each  regional  commissioner  is 
assisted  by  state  commissioners  in  carrying 
out  the  operations  within  individual  states. 
The  state  commissioner's  duties  may  vary 
according  to  the  responsibilities  delegated 
by  the  regional  commissioner,  but  in  most 
cases,  his  major  responsibility  is  to  assist 
in  the  recruiting,  training  and  assigning  of 
inspectors . 
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Each  inspector  is  a  board-certified 
pathologist  who  has  attended  a  recent  inspec- 
tor's workshop  or  received  on-the-job  train- 
ing during  an  inspection  with  an  experienced 
inspector.     Each  inspector  is  assisted  by  a 
team  of  specialists,  chosen  by  the  patholo- 
gist-inspector, based  on  the  size  and  complex- 
ity of  the  lab  which  they  are  assigned  to 
inspect.     For  example,  in  a  two  or  three 
hundred  bed  community  hospital,  the  inspector 
might  select  a  chemist,  a  chief  technologist 
and  possibly  a  supervisory  technologist.  In 
a  university  setting  the  team  may  include 
several  other  pathologists  and  a  number  of 
doctoral  level  assistants.     The  team  is  usu- 
ally chosen  from  within  the  inspector's  own 
institution,  but  this  is  only  because  of  the 
logistics  involved  in  recruiting  inspectors. 
We  are  currently  developing  a  computer  pro- 
gram to  provide  a  data  base  on  inspectors, 
and  hopefully,  this  will  make  it  easier  for 
an  inspector  to  recruit  assistants  from  other 
laboratories  in  his  or  her  area. 

The  commissioners  and  inspectors,  through 
daily  contact  with  the  many  laboratories  in 
the  program,  are  continuously  aware  of  the 
expanding  technology  and  changing  needs  within 
the  practice  of  laboratory  medicine.  Inqui- 
ries are  addressed  to  the  College  from  lab- 
oratories all  over  the  country,  questioning 
the  scientific  relevance  of  certain  require- 
ments, or  seeking  advice  on  acceptable  meth- 
ods for  meeting  the  requirements.     All  of 
these  inquiries  are  reviewed  by  the  commis- 
sion and  are  often  referred  to  one  of  the 
College's  resource  committees.     The  resource 
committees  are  comprised  of  pathologists  with 
expertise  in  special  areas  such  as  toxicology, 
parasitology  and  so  on.     The  resource  commit- 
tees review  the  questions  referred  by  the 
commission  and  offer  their  expert  knowledge 
in  resolving  these  issues.     In  many  cases, 
the  questions  raised  by  the  participants  or 
inspectors  point  up  outdated  aspects  of  the 
program,  which  are  then  promptly  changed. 
Such  input  also  adds  to  the  further  develop- 
ment of  the  program  by  calling  attention  to 
details  that  might  ordinarily  be  overlooked. 

3.     Standards  and  Criteria  for 
Accreditation 

The  standards  required  to  achieve 
accreditation  are  very  comprehensive, 
addressing  all  aspects  of  quality  control, 
such  as  methodology,  reagents,  control 
media,  equipment,  proficiency  testing,  speci- 
men handling,  and  reports.     They  also  include 
requirements  on  safety,  instrument  mainte- 
nance, personnel,  laboratory  environment  and 
facilities.     In  order  to  assure  that  all  of 
these  requirements  are  met  uniformly  in  each 


laboratory,  the  commission  has  developed 
Inspection  Checklists  containing  more  than 
1700  questions  on  all  areas  of  the  lab. 
Today's  checklists  are  computerized  to  alio 
for  constant  updating,  as  the  state-of-the-|: 
art  continues  to  expand.     These  checklists 
serve  as  an  aide  to  the  inspector,  since 
there  are  so    many  requirements,  and  it  is 
very  possible  to  overlook  something  without 
such  a  guide.     However,  in  a  peer-review  pr 
gram,  the  real  emphasis  is  on  the  profes- 
sional judgement  of  the  inspector.  Through 
his  knowledge  of  the  I&A  Standards  and  his 
experience  as  a  pathologist,  the  inspector 
evaluates,  not  only  the  routine  systems  of 
quality  control  within  the  laboratory,  but  Jk 
also  the  effectiveness  of  various  approache, 
the  underlying  attitudes  and  the  management 
principles  which  determine  how  well  the  lab 
oratory  is  serving  the  patient. 


k.     Proficiency  Testing 

One  of  the  major  requirements  for 
achieving  and  maintaining  accreditation,  is 
the  successful  participation  in  an  interlab 
oratory  comparison  program.     Laboratories  i 
the  College's  ISA.  Program  are  required  to 
participate  in  the  proficiency  testing  pro- 
grams offered  by  the  College.     The  College 
refers  to  its  proficiency  testing  programs 
as  Survey  Programs  (not  to  be  confused  with 
inspection).     The  College  Survey  Program  is 
the  most  widely  accepted  program  of  its  kin 
with  more  than  9000  laboratories  enrolled, 
providing  the  largest  data  base  for  inter- 
laboratory  comparison  in  the  world.     At  the 
time  of  the  on-site  inspection,  the  profi- 
ciency test  results  are  reviewed  by  the 
inspector.     The  regional  commissioner  again 
reviews  the  results  as  part  of  the  overall 
review  of  the  inspection  report.     All  unac- 
ceptable results  (those  outside  the  limits 
established  by  the  comparison)  must  show 
evidence  of  review  and  corrective  action. 
Once  the  laboratory  is  accredited,  a  semi- 
annual summary  of  survey  results  is  gener- 
ated by  the  computer  and  sent  to  the  lab. 
Copies  of  this  report  are  also  sent  to  the 
regional  commissioner  for  review.  This 
assures  that  an  accredited  laboratory  is 
continuously  being  monitored  for  proficienc; 
in  specimen  identification. 

5.  Education 

Continuing  education  through  peer- 
review  and  professional  consultation  is  an 
integral  part  of  a  program  such  as  the 
College's  I&A  Program.     The  inspector  and 
members  of  the  team  bring  with  them  the 
knowledge  and  experience  they  have  gained 
through  past  inspections,  as  well  as  their 
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ywn  professional  experiences.     This  provides 
-he  setting  for  an  exchange  of  knowledge  that 
„s  beneficial  for  both  the  laboratory  being 
>->e  viewed  and  those  performing  the  review. 
•"In  recognition  of  this  fact,  the  American 
fedical  Association  has  authorized  the  issu- 
ance of  continuing  medical  education  credit 
to  pathologists  practicing  in  CAP  accredited 
Labs  and  to  each  pathologist  who  inspects 
?Tor  the  College.     One  of  the  best  examples 
jf  the  educational  process  involved  is  seen 
| La  the  summation  conference.     During  this 
conference  the  inspection  team  meets  with 
.she  laboratory  director  and  the  staff  to  go 
over  the  findings  of  the  inspection.  The 
Inspector's  Manual  [l]  that  is  provided  to 
8.; he  inspector  each  time  he  agrees  to  perform 
an  inspection,  states  that  "The  summation 
Conference  is  an  appropriate  time  to  present 
she  objectives  of  the  inspection  and  accred- 
itation program  and  discuss  improvement  of 
'she  laboratory  in  educational  terms." 

6.  Government  and  Professionel 

Relations 

There  are  many  programs ,  both  govern- 
mental and  professional,  that  conduct  inspec- 
tions in  the  lab.     For  many  years,  the  Col- 
lege has  worked  to  eliminate  duplicate 
inspections  through  close  liaison  with  other 
■ijgroups.     The  College  has  maintained  an  equiv- 
alency with  the  Clinical  Laboratory  Improve- 
ment Act  of  1967  since  its  implementation. 
This  equivalency  enables  a  laboratory  accred- 
ited by  the  College  to  apply  for  an  exemption 
sfrom  licensure  and  to  substitute  its  College 
inspection  in  lieu  of  federal  inspection. 
|;The  College  also  had  formal  agreement  with 
'three  states,  Georgia,  Connecticut,  and 
(Tennessee,  to  substitute  CAP  inspection  for 
jstate  inspection  and  negotiations  are  cur- 
rently underway  in  several  other  states.  In 
addition,  the  College  has  recently  entered 
into  an  agreement  with  the  Joint  Commission 
on  Accreditation  of  Hospitals,  whereby  hos- 
[pital  based  laboratories  accredited  by  CAP 
are  not  inspected  by  JCAH.     The  College  also 
maintains  liaison  with  many  of  the  specialty 
societies  within  the  laboratory  field,  such 
as  the  American  Association  of  Blood  Banks 
and  the  American  Association  of  Clinical 
Chemists . 

7.  Inspection  Process 

A  laboratory  achieves  accreditation 
from  the  College  through  the  following  steps: 
First  they  phone  or  write  our  office  to 
request  an  application.     We  send  the  lab  a 
packet  of  material  consisting  of  a  Long 
[Form  Questionnaire,  the  Inspection  Check- 
lists, which  I  showed  you  earlier,  a  booklet 


of  Standards  and  a  covering  letter  instruct- 
ing the  lab  on  how  to  complete  the  informa- 
tion required.     The  Long  Form  Questionnaire 
contains  questions  on  test  volume,  instru- 
mentation, methodology,  and  quality  control. 
We  also  ask  that  the  lab  provide  a  personnel 
form  on  all  professional  and  supervisory 
personnel,  a  sketch  of  the  laboratory  and 
copies  of  proficiency  test  results  for  the 
past  year.     This  data  provides  information 
to  the  inspector  before  his  visit,  which 
indicates  what  will  be  involved  in  the 
inspection.     In  this  way  he  can  also  select 
an  appropriate  team.     It  also  supplies  infor- 
mation to  our  office  which  is  kept  in  a 
permanent  file . 

The  checklists  give  the  lab  a  chance  to 
review  the  requirements  in  each  section  be- 
fore submitting  the  forms  for  inspection. 
Since  the  standards  are  stringent,  it  often 
takes  several  months  for  the  laboratory  pre- 
paring for  its  first  inspection  to  institute 
all  of  the  quality  control  measures  that  are 
required  for  accreditation.     Procedure  man- 
uals need  to  be  updated,  documentation  re- 
viewed, and  checks  made  for  health  records, 
safety  manuals  and  maintenance  records. 
Often  the  lab  preparing  for  its  initial 
inspection  will  send  a  representative  to 
one  of  the  Laboratory  Improvement  Seminars 
that  are  presented  throughout  the  year  by 
the  commission. 

Once  the  completed  forms  are  received 
in  our  office,  a  notice  is  sent  to  the  com- 
missioner informing  him  that  the  lab  is 
ready  for  inspection.     The  commissioner 
then  appoints  an  inspector  to  visit  the 
laboratory  and  notifies  us  so  that  we  can 
send  the  data  on  the  lab  to  the  inspector. 
The  inspector  reviews  the  data  and  contacts 
the  lab  to  arrange  for  an  inspection  date. 
All  inspection  visits  are  announced.  The 
inspection  is  normally  scheduled  about  five 
weeks  from  the  time  the  lab  submits  the 
completed  forms. 

The  inspector  and  the  team  then  visit 
the  laboratory.     They  hold  a  preliminary 
orientation  meeting  with  the  lab  director 
and  department  heads.     In  a  hospital  lab, 
the  inspector  also  meets  with  the  hospital 
administrator  and  chief  of  staff.  The 
inspection  team  then  goes  through  the  lab 
and  completes  the  inspection  checklists. 
After  the  inspection,  the  inspection  team 
and  laboratory  staff  assemble  for  the  sum- 
mation conference,  mentioned  earlier. 

After  the  on-site  visit,  the  inspector 
sends  his  report  to  the  College  computer 
center  where  a  computerized  list  of 
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deficiencies  is  generated.     This  computer 
report,  along  with  the  inspector's  written 
comments  is  sent  to  the  regional  commissioner 
with  the  rest  of  the  file.     The  commissioner 
then  reviews  the  report  and  sends  a  copy  to 
the  lab  director  with  appropriate  comments, 
requesting  that  the  deficiencies  he  corrected 
and  that  documentation  be  provided  for  each 
corrective  action.     The  lab  is  given  30  days 
to  respond  to  the  report. 

Once  the  commissioner  receives  the 
response  from  the  lab,  he  carefully  reviews 
the  entire  report,  and  once  he  is  satisfied 
that  each  deficiency  has  been  adequately  cor- 
rected and  documented,  he  sends  his  recom- 
mendation to  accredit  the  lab  to  our  office, 
and  an  accreditation  letter  is  sent  to  the 
laboratory  under  the  chairman's  signature. 
A  certificate  is  also  ordered  at  this  time. 
Formal  approval  is  made  by  the  entire  com- 
mission at  its  next  regularly  scheduled 
meeting.     If  the  lab  fails  to  meet  the  stan- 
dards required  for  accreditation,  it  is  ad- 
vised on  ways  to  correct  the  deficiencies 
and  invited  to  re-apply  after  one  year. 
Each  year  there  are  approximately  twelve  to 
sixteen  labs  that  are  denied  accreditation. 

Each  step  of  this  process  is  monitored 
on  our  computer  and  a  weekly  report  is  sent 
to  our  office  so  that  we  can  keep  track  of 
all  the  labs  being  inspected.     Once  the  lab- 
oratory is  accredited,  this  information  is 
entered  on  the  computer  and  the  time  is 
automatically  set  so  that  the  lab  receives 
reinspection  forms  90  days  before  its  accred- 
itation is  due  to  expire.     The  accreditation, 
by  the  way,  is  valid  for  a  two  year  period. 
During  the  interim  year,  between  on-site 
visits,  the  lab  is  sent  a  set  of  the  Inspec- 
tion Checklists  and  asked  to  perform  a  self 
inspection.     This  enables  the  lab  to  evalu- 
ate its  current  status  and  correct  any  defi- 
ciencies that  may  have  cropped  up  since  the 
last  inspection.     A  copy  of  the  self  inspec- 
tion report  is  sent  to  the  inspector  who  is 
assigned  to  perform  the  next  on-site  visit, 
so  that  he  can  check  to  see  that  the  defici- 
encies noted  in  the  interim  year  have  been 
corrected. 
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Our  data  shows  that  each  subsequent 
inspection  uncovers  fewer  and  fewer  defici- 
encies.    This  is  a  clear  indication  that  the 
program  is  meeting  its  primary  goal  of  lab- 
oratory improvement. 
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LABORATORY  ACCREDITATION  FOR 
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Certification  of  toxicologists  and  accreditation  of  toxicology 
laboratories  have  been  the  subject  of  concern  to  the  Society  of 
Toxicology  since  1973.     Numerous  committees  have  studied  the  problem 
and  all  came  to  the  conclusion  that  such  programs  would  greatly 
enhance  the  science  of  toxicology.     This  paper  describes  certification 
and  accreditation  programs  for  toxicologists  and  toxicology  labora- 
tories . 

Key  words:     Accreditation;  certification  criteria;  evaluation; 
toxicology  laboratories. 


1.  Introduction 

For  some  years,  there  has  been  concern 
on  the  part  of  the  regulatory  agencies  in  the 
Federal  government  about  the  standards  of 
performance  of  laboratories  submitting  data 
on  chemicals  that  must  receive  approval  prior 
to  shipment  in  interstate  commerce.  Congres- 
sional committees  and  consumer  organizations 
have  been  quick  to  blame  toxicologists  for 
having  inadequate  performance  standards. 
!  When  certain  laboratories  were  charged  with 
failure  to  meet  good  laboratory  practices, 
the  credibility  of  toxicologists  suddenly 
came  under  fire.     Officials  of  the  Food  and 
Drug  Adminstration  and  the  Environmental 
Protection  Agency  testified  before  Congres- 
sional committees  that  data  from  toxicology 
laboratories  could  not  be  relied  upon  and 
something  would  have  to  be  done  to  ensure 
the  quality  of  performance  by  laboratories 
submitting  data  to  regulatory  agencies.  The 
Society  of  Toxicology  became  deeply  concerned 
about  these  charges  and  responded  by  support- 
ing and  sponsoring  a  certification  and  ac- 
creditation program. 


2.  Certification 

In  June,  1979,  the  American  Board  of 
Toxicology  was  incorporated  in  the  District 
of  Columbia.     It  is  now  in  the  process  of 
preparing  a  "self-evaluation  examination" 
program  which  will  be  available  to  anyone 
who  wishes  to  test  his  or  her  skills  on  com- 
petence in  toxicology.     The  examination  will 
be  graded  and  persons  found  to  be  deficient 
in  any  area(s)  will  then  have  an  opportunity 
to  take  refresher  course(s)  in  preparation 
for  a  full-fledged  examination  for  certifi- 
cation.    The  response  to  participate  in  this 
program  has  been  most  encouraging  and  indi- 
cates that  toxicologists  regard  toxicology 
as  requiring  a  high  degree  of  professional- 
ism.    The  Board  is  totally  independent  of 
the  Society  of  Toxicology  and  elects  its  own 
officers  and  Directors.     It  did  however  re- 
ceive substantial  financial  support  from  the 
Society  to  help  defray  expenses  in  getting 
the  program  underway. 
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3.  Accreditation 

The  development  of  reliable  toxicologi- 
cal  information  depends  upon  the  caliber  of 
the  laboratories  from  which  basic  data  are 
obtained. 

In  July,  1979,  the  Toxicology  Labora- 
tory Accreditation  Board  was  incorporated 
in  the  District  of  Columbia  to  provide  a 
program  for  laboratory  accreditation.  The 
Board  consists  of  twelve  members,  with  ap- 
proximately equal  representation  from  in- 
dustry, government,  academia,  and  consulting 
laboratories.     The  Board  is  independent  of 
the  Society  of  Toxicology,  but  received  a 
grant  in  support  of  the  organizational  work. 

The  purpose  of  the  Toxicology  Labora- 
tory Accreditation  Board  is  to  encourage, 
promote,  and  maintain  high  standards  of  per- 
formance in  toxicology  through  a  system  of 
accreditation  by  (l)  recognizing  those  lab- 
oratories demonstrating  competence  in  vari- 
ous aspects  of  toxicology;  and  (2)  promot- 
ing good  laboratory  practices  in  toxicology. 
The  Board  will  accredit  a  toxicology  labora- 
tory in  specific  functional  areas  for  which 
the  laboratory  requests  accreditation  and 
demonstrates  that  it  meets  the  standards 
and  requirements  for  accreditation.  Any 
laboratory  applying  for  accreditation  must 
designate  on  the  application  those  func- 
tional areas  for  which  accreditation  is  de- 
sired.    For  each  area,  the  toxicology  lab- 
oratory may  receive  full  accreditation, 
provisional  accreditation,  or  accreditation 
withheld.     To  receive  full  accreditation,  an 
applicant  must  have  completed  and  prepared 
reports  or  publications  of  studies  in  the 
functional  area  for  which  accreditation  is 
sought.     Failure  to  obtain  full  accredita- 
tion in  one  functional  area  will  not  nor- 
mally influence  the  action  of  the  Board  in 
considering  other  areas  within  the  same 
laboratory.     A  list  will  be  published  an- 
nually, specifying  functional  areas  in  which 
full  accreditation  has  been  granted.  Pro- 
visional accreditation  will  be  granted  to  a 
laboratory  which  meets  all  requirements  in 
staff,  facilities  and  procedures  but  has  not 
demonstrated  proficiency.     Within  a  limited 
period  of  time  (to  be  determined  by  the 
Board),  the  laboratory  must  meet  the  defi- 
ciency provision  to  become  fully  accredited 
or  provisional  accreditation  will  be  re- 
voked. 

A  laboratory  that  has  been  fully  ac- 
credited and  subsequently  found  to  have 
serious  deficiencies  will  be  notified  of 
those  deficiencies  and  given  the  status  of 
probationary  accreditation .     Within  a 
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specified  period  of  time  (to  be  determined 
by  the  Board)  the  deficiencies  must  be  cor 
rected  or  accreditation  will  be  revoked. 
Laboratories  from  which  accreditation  is 
withheld  will  be  given  a  report  listing  th 
deficiencies  and  the  laboratory  may  reappl 
(no  earlier  than  six  months)  before  consid 
eration  will  be  given  for  accreditation. 
Laboratories  applying  for  accreditation  wi 
be  required  to  pay  a  nonrefundable  applies 
tion  fee,  the  amount  being  determined  at  t 
time  of  application,  the  size  of  the  labor 
tory,  number  of  professional  and  technical 
staff,  number  of  animals  housed,  and  an  es 
mate  of  the  time  required  for  the  site  vis 
To  help  defray  expenses  of  the  Board,  accr 
ited  laboratories  will  be  required  to  pay 
an  annual  fee . 

It  is  anticipated  that  toxicology  lab 
oratories  will  seek  accreditation  for  the 
following  reasons:     (l)  to  serve  as  eviden 
that  an  accrediting  organization  of  profes 
sional  peers  considers  the  laboratory  cap- 
able of  performing  toxicological  investi- 
gations,  (2)  to  provide  outside  consultati 
with  the  assurance  that  the  procedures  to 
followed  are  consistent  with  the  "state  of 
the  art"  and,   (3)  to  serve  as  a  means  of 
determining  whether  facilities  and  equipme 
or  procedures  could  be  improved  upon . 

The  Board  is  currently  in  the  process 
of  preparing  a  manual  on  procedures  for 
accreditation  which  will  include  instruc- 
tions for  filing  applications,  payment  of 
fees,  and  preparations  for  site  visits. 
The  site  visit  team  will  consist  of  two  or 
more  individuals,  including  experts  in  the 
area  for  which  accreditation  is  being  song 
The  review  will  be  based  on  guidelines  for 
good  laboratory  practices,  as  established 
by  the  Board.     Particular  attention  will  h 
given  to  such  things  as  qualifications  of 
personnel,  adequacy  of  facilities  and  equi; 
ment,  scientific  soundness  of  protocols, 
conduct  of  study,  appraisal  of  the  results 
and  the  program  for  quality  assurance. 

It  is  hoped  that  the  establishment  of 
the  Toxicology  Laboratory  Accreditation 
Board  will  lessen  the  need  for  government 
intervention.     It  is  essential  that  person: 
working  in  laboratories  be  given  freedom  t< 
pursue  new  ideas  while  at  the  same  time  wo: 
under  self-imposed  high  standards.  The 
Accreditation  Board  fulfills  this  need  be- 
cause it  is  a  voluntary  program. 
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LABORATORY  PERFORMANCE  EVALUATION  AND  ACCREDITATION  - 
IT'S  ALL  IN  THE  IMPLEMENTATION 

WILLIAM  J.  SMITH  AND  G.  THOMAS  CAS  UNO 

UNDERWRITERS   LABORATORIES  INC. 
NORTHBROOK,    ILLINOIS  60062 


Based  on  many  years  of  experience  in  the  develop- 
ment of  product  evaluation  information  under  Federal, 
State  and  local  Codes  and  Standards  and  related  in-house 
studies  of  associated  accreditation  programs,   the  criti- 
cal features  of  these  programs  have  been  identified  in 
terms  of  the  degree  to  which  they  are  likely  to  achieve 
their  objectives.     Most  accreditation  programs  place 
heavy  emphasis  on  written  documentation  of  conformance 
to  established  criteria  and  in-house  quality  assurance 
programs,   stressing  test  equipment  calibration  and 
maintenance,  with  limited  attention  to  the  critical  con- 
trol features  and  measures  of  the  extent  of  in-house 
implementation.     A  review  of  the  critical  features  of 
laboratory  accreditation  programs  indicates  that  these 
programs  must  include:    (1)    "hands-on"  proficiency  of 
personnel,    (2)   feedback  mechanisms  for  program  improve- 
ment,   (3)  procedures-in-case-of  non-compliance  and/or 
errors,    (4)  random  unannounced  inspections  of  test 
facilities  and   (5)   independent  countercheck  reference 
testing.     Accreditation  programs  lacking  in  one  or  more 
of  these  critical  features  can  drift  to  a  "least  common 
denominator"  modus  operandi  and  fall  short  of  original 
objectives . 

Key  words:     Countercheck  reference  tests;  critical 
control  features;   feedback;  implementation; 
inspections;   laboratory  accreditation;   laboratory  per- 
formance evaluation;  noncompliance;  proficiency. 


Introduction 

Virtually  all  laboratory  accredi-  stressing  test  equipment  calibration 

ration  programs  currently  sponsored  and  maintenance.     In  most  cases 

:>y  Federal,   State,   and  local  regula-  limited  attention  is  given  to 

:ors  place  heavy  emphasis  on  written  critical  control  features  of 

Jocumentation  of  conformance  to  laboratory  quality  assurance  pro- 

?stablished  criteria  and  in-house  grams  which,   if  given  higher 

quality  assurance  programs,  generally  priority  in  evaluating  the 
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performance  of  individual  programs 
would  provide  for  an  overall  higher 
level  of  accuracy  and  uniformity  of 
test  results  and/or  product  certifi- 
cations . 

We  believe  that  a  review  of  key 
elements  of  laboratory  evaluation 
and  accreditation  programs  is  neces- 
sary to  fully  appreciate  the 
significance  of  the  phrase:  "It's 
all  in  the  implementation."  These 
include:    "hands-on"  proficiency  of 
personnel,   feedback  mechanisms  for 
program  improvement,  procedures-in- 
case-of  non-compliance  and/or  errors, 
the  importance  of  random  unannounced 
inspections  of  test  facilities  and 
the  need  for  independent  counter- 
check reference  testing. 

Consideration  must  be  given  to 
current  laboratory  accreditation 
programs  that  appear  to  be  drifting 
into  the   "least  common  denominator" 
modus  operandi  with  accredited 
laboratories  marginally  satisfying 
established  criteria.  Accredited 
laboratories  unwilling  to  lower 
their  performance  standards  to  this 
"least  common  denominator"  can,  by 
choice,   restrict  their  involvement 
in  particular  areas  of  product  test- 
ing and  certification  or  withdraw 
from  the  entire  accreditation  pro- 
gram.    Emphasis  upon  implementation 
of  the  essential  criteria  identified 
as  a  result  of  these  considerations 
must  be  stressed  in  order  to  minimize 
inequity  and  improve  the  overall 
effectiveness  of  any  laboratory 
accreditation  program. 

Studies  of  Accreditation  Programs 

Underwriters  Laboratories  is 
currently  involved  in  a  number  of 
laboratory  accreditation  programs 
covering  a  wide  spectrum  of  products 
and  services.     This  involvement  has 
provided  opportunities  to  study  the 
key  features  of  each  program  and 
evaluate  strengths  and  weaknesses. 
These  programs  are  all  related  to 


UL's  activities  concerned  with  the 
evaluation  and  testing  services  of 
materials,  products,  and  systems 
for  public  safety.     These  programs 
are  sponsored  or  managed  by  all 
segments  of  government  —  Federal,  ; 
State,   and  municipal  —  and,   in  soznf  ? 
cases,  by  private  organizations. 

\ 

The  High  Velocity-Short  Range  Progra i 

i 

The  nature  of  accreditation 
criteria  and  their  implementation 
vary  greatly  among  programs.  Some 
are  limited  to  the  most  basic 
elements  necessary  to  accredit  a 
facility:  place  of  business,  chief] 
person  to  contact  at  the  facility, 
financial  status,   staff  size,  and 
the  nature  of  services  provided, 
accompanied  with  a  prepaid  fee.  In 
these  cases,   the  accreditation 
process  is  simple,   and  limited  in 
scope.     Problems,    if  any,   usually   i:  "ri 
relate  to  some  minuscule  detail 
overlooked  in  the  application 
process.     We  refer  to  these  progran 
as   "High  Velocity-Short  Range" 
programs . 

The  Paper  Based-Middle  Range  Progra'  ~ 

Other  accreditation  programs,  ■■  '■■ 
however,   are  supported  by  a  set  of"  -: 
"rules  and  regulations"  or  accredi-1 
tation  criteria,   varying  in  detail,5  :i 
and  covering  a  broad  range  of  info]'  !' 
mation  about  the  operation  and 
qualifications  of  the  laboratory 
seeking  accreditation.  Ownership 
and  management  structure,  resumes 
of  key  personnel  and  their  report-  I 
ing  relationships,  details  of  the   1  :; 
physical  plant  and  test  equipment, '  : 
record  keeping,   report  format  and 
in-house  review  procedures,   certi-  I 
fication  mark  or  label  control, 
feedback  and  corrective  action  pro' 
cedures,   test  equipment  standardi-  ^ 
zation  and  calibration,  laboratory 
quality  control  procedure  and  free-'  ;: 
dom  from  conflict  of  interest  are 
among  the  details  usually  included 
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.n  these  criteria.     Applications  for 
accreditation  usually  must  be  accom- 
janied  by  voluminous  paper  detail, 
locumenting  or  responding  to,   in  one 
form  or  another,   the  established 
criteria.     Accreditation  under  these 
:ypes  of  programs,  what  we  term  "Paper 
iased-Middle  Range"  programs,  is 
>ased  primarily  upon  a  review  of  the 
^aper  submitted  by  the  applicant  and 
:he  prepayment  of  fees.     Again,  any 
iroblems  usually  relate  to  overlooked 
>aperwork  detail. 


These  paper  based  programs  are 
lumerous;  an  example  which  typifies 
hem  occurred  several  years  ago.  UL 
tade  application  for  accreditation 
nder  the  designated  Rules  and 
egulations.     On  the  surface,  the 
•rogram  appeared  to  be  detailed, 
l|[|road  scoped  and  comprehensive.  We 
kroceeded  with  some  apprehension  as 
o  whether  or  not  we  could  readily 
bet  all  the  criteria  that  had  been 
stablished  while  at  the  same  time 
jeing  pleased  that  the  authors  of 
rjjcjhe  program  showed  such  initiative  in 
Ijhe  development  of  comprehensive 
■uality  assurance  criteria  for 
ccreditation  for  all  applicants.  We 
ade  considerable  effort  to  gather 
q^jll  the  voluminous  paper  required  to 
espond  to  the  criteria  and  furnished 
:he  several  copies  required  along 
:ith  the  accreditation  fee.  Subse- 
quently,  after  one  or  two  trips  to 
[he  accrediting  organization  office 
f Jo  discuss  certain  details,  we  were 
ccredited  and  began  to  implement  our 
ervices  under  that  accreditation, 
ased  upon  the  number  of  hurdles  we 
|ad  overcome  to  gain  accreditation 
nitially,  we  fully  expected  a  con- 
jinuing  and  fruitful  dialogue  with 
(he  accreditation  program  administra- 
ors  concerning  the  implementation 
If  the  program.     Instead,  our  total 
nvolvement  with  the  administrators 
or  the  past  2-3  years  has  been  an 
nnual  letter  reminding  us  that  con- 
inued  accreditation  was  contingent 
m  the  payment  of  the  renewal  fee. 
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Multi-Faceted-Broad  Range  Programs 

Finally,   there  are  accredita- 
tion programs  that  involve  the  full 
range  of  elements  including:   a  com- 
prehensive application  to  establish 
qualifications,   initial  and  follow- 
up  on-site  inspections,   a  required 
in-house  quality  assurance  program, 
proficiency  testing  and  at  least 
an  announced  intent  at  the  equit- 
able implementation  of  the 
originally  established  accredita- 
tion criteria,   including  deaccredi- 
tation  procedures. 

These  are  the  types  of  programs 
that  must  serve  as  the  foundation 
upon  which  the  laboratory  accredita- 
tion programs  are  built,   if  indeed 
the  overall  concept  is  to  succeed. 
In  fact,   it  is  only  through  the  use 
of  the  multi-faceted-broad  range 
approach  that  the  overall  need  for 
the  establishment  of  individual 
laboratory  accreditation  programs 
can  be  supported.     If  high  velocity- 
short  range  or  paper  based-middle 
range  programs  continue  to  be  the 
trend,   the  entire  concept  of  labora- 
tory accreditation  will  no  longer 
be  supportable. 


The  Accreditation  Panacea 

Many  accreditation  programs 
depend  upon  the  occurrence  of  non- 
conformances in  the  field  to 
trigger  actions  by  the  accreditor 
to  seek  correction  on  the  part  of 
one  or  more  participating  accredited 
laboratories,  or  to  begin  deaccredi- 
tation  proceedings.     In  many 
respects  this   "de  facto  accredita- 
tion" system  has  merit.     With  one 
essential  difference  it  is  this 
type  of  system  which  the  work  of 
third  party  laboratories  has  been 
judged,  accepted  for  many  years  and 
continues  to  be  judged  and  accepted 
in  numerous  product  testing  areas, 
long  before  formal  accreditation 
programs  came  into  vogue.     It  is 
the  system  under  which  UL  has  grown 
and  gained  National  and  Inter- 
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national  acceptance.     The  essential 
difference  in  the  emerging  system  of 
accreditation  is  that  participating 
laboratories  initially  receive  an 
aura  of  high  competence  and  perfor- 
mance capability  under  a  cloak  of 
accreditation.     Whether  or  not  the 
aura  is  deserved  or  later  grows 
depends  upon  the  nature  of  the 
accreditation  process.     A  super- 
ficial and  ineffective  accreditation 
program  can  lead  to  acceptance  of 
the  "least  common  denominator" 
modus  operandi,  wherein  the  accred- 
ited laboratory  marginally  satisfying 
the  criteria  sets  the  level  of  per- 
formance toward  which  laboratories 
accredited  under  the  program  migrate 
or  feel  pressure  to  follow  the  down- 
ward trend.     The  sinking  ship 
analogy  may  fit  in  this  situation; 
one  either  elects  to  say  on  board 
and  ride  it  down  with  the  others, 
or  take  to  the  lifeboats,  salvaging 
what  you  can  in  the  process. 


Critical  Control  Features 

Based  upon  many  years  of 
experience  and  related  in-house 
studies,   it  is  recommended  that  five 

"critical  control  features"  be 
included  in  laboratory  evaluation 
and  accreditation  programs  and 
implemented  in  conjunction  with  such 
programs.     These  features  are: 

(1)    "hands-on"  proficiency  of  person- 
nel,   (2)   feedback  mechanisms  for 
program  improvement,    (3)  procedures- 
in-case-of  non-compliance  and/or 
errors,    (4)  random  unannounced  inspec- 
tions of  test  facilities  and  (5) 
independent  countercheck  reference 
testing.     While  each  of  these 
elements  is  individually  treated  in 
greater  detail  herein,   it  is  vital 
that  they  be  implemented  collect- 
ively; only  in  this  way  will  the 
overall  benefit  to  a  given  program 
be  realized  through  the  synergistic 
effect  produced. 


(1)     Hands-on  Proficiency  of 
Personnel  -  The  present  emphasis 

82 


on  evaluation  and  accreditation  of 
laboratories  based  upon  review  of 
paper  documentation  of  qualif icatioi 
should  be  reversed.     On-site  evalua- 
tion should  include  the  witnessing 
of  the  conduct  of  tests  or  the  per-: 
formance  of  service  by  those  person 
nel  that  are  actually  engaged  in  th< 
work.     Objective  evaluations, 
including  oral  interviews  designed 
to  elicit  the  level  of  familiarity 
with  the  test  method  and  equipment 
should  be  held  with  involved  person 
nel  at  both  supervisory  and  technic 
levels . 


(2)     Feedback  Mechanisms  for 
Program  Improvement  -  Few  existing 
accreditation  programs  include 
mechanisms  for  feedback  from  other 
laboratories  or  users  of  the 
product  or  service.     Such  mechanism 
should  be  developed  and  their  use 
encouraged.     Feedback  mechanisms 
could  include  a  system  for  categori 
ing  and  analyzing  complaints,  perio 
seminars  with  accredited  laboratory 
personnel  and  users,  periodic  infor 
mational  letters  or  bulletins  to 
participants  that  call  attention  to 
any  emerging  sub-standard  features 
of  the  program  with  recommendations 
or  directives  for  corrective  actior 
and  frequent  in-depth  analysis  by 
the  administrators,  with  input 
solicited  from  the  participants,  tc 
determine  if  the  program  is  meetinc 
its  objectives.     To  be  successful, 
however,   the  utilization  of  feedbac 
mechanisms  must  be  for  the  purpose 
of  improving  the  program,  rather 
that  solely  being  the  source  of 
implementing  punitive  actions. 


(3 )     Procedure-In-Case-Of  Non 


■ 


Compliance  or  Errors  -  This  control^- 
feature  is  perhaps  of  the  greatest 
overall  importance  to  the  effective 
ness  of  an  accreditation  program. 
If  the  actions  taken  to  correct  a 
non-compliance,  either  under  the 
accreditation  program  or  the  in- 
house  quality  assurance  programs  o: 
the  participating  laboratories, 
represent  long-term  procedures 


i/herein  the  non-compliance  (i.e., 
aboratory  program  defect,  incorrect 
est  data  generation,  etc.)  is 
1 lowed  to  continue  until  a  resolu- 
ion  is  found  or  on  a  so-called 


emporary  provisional  basis,  wh ich 


n  reality  becomes  a  permanent  con- 
lition,   then  meaningful  procedures 

n-case-of  non-compliance  exist  in 
lame  only. 

Effective  procedures-in-case- 
>f  non-compliance  are  based  upon 
:he  premise  that  actions  associated 
'ith  a  proven  non-compliance  must 
ease.     This  means,   for  example, 
hat  an  accredited  laboratory  ceases 
o  conduct  certain  tests  under  the 
lanner  of  the  accreditation  until 
he  non-compliance  is  resolved;  this 
leans  that  manufacturers  of  the 
roducts  so  tested  cease  using  the 
on-complying  product  performance 
est  data  in  conjunction  with  the 
arketing  of  the  product;   this  means 
hat  if  the  non-compliance  continues, 
eaccreditation  follows  expediciously . 

Indeed  the  strength  and  substance 
f  the  answer  to  the  question  "what 
appens  in  case  of  non-compliance?" 
epresents  the  strength  and  sub- 
tance  of  the  entire  accreditation 
rogram.     This  question  must  be 
sked  of  all  participants  in  any 
ccreditation  program  and  the 
nswer  carefully  analyzed  by  the 
ccrediting  organization.  Organiza- 
ions  utilizing  multi-faceted  broad 
ange  accreditation  programs  will 
ecognize  the  weak,  non-substantive 
nswers  as  being  characteristic  of 
neffective  in-house  laboratory 
ality  assurance  programs. 

(4)     Unannounced  Inspection  of 
ist  Facilities  -  It  is  recognized 


hat  the  initial  on-site  evaluation 
f  personnel  and  test  equipment 
hould  be  announced.  Laboratories 
ay  need  a  reasonable  amount  of  time 
o  determine  that  the  facilities  are, 
n  fact,   ready  for  accreditation 
nder  particular  criteria  of  a  given 
rogram.     Once  accredited  and 


providing  services  under  the  program, 
laboratories  should  be  subject  to 
random,   unannounced  on-site  inspec- 
tions.    They  serve  as  a  means  for 
early  detection  of  substandard  per- 
formance, disclosure  of  major 
changes  in  key  personnel     and  signi- 
ficant variations  in  major  test 
equipment  and  serve  as  a  counter- 
check that  important  recommendations 
for  improvement  have  been  imple- 
mented.    Most  importantly,  this 
unannounced  visit  permits  the 
examiner  to  make  a  more  accurate 
evaluation  of  the  laboratories 1 
day-to-day  operations  and  avoid  one- 
time demonstrations  capability  that 
can  be  associated  with  announced 
facility  inspections. 

(5 )     Independent  Countercheck 
Reference  Testing  -  Another  very 
effective  mechanism  for  monitoring 
the  work  of  laboratories  is  to 
continually  subject  the  results  of 
a  wide  spectrum  of  their  work  to 
review  and  countercheck.  Anonymity 
among  participating  laboratories  is 
a  necessary  requirement,  as  is  the 
need  for  reporting  the  results  to 
participants.     Given  the  oppor- 
tunity to  compare  the  results  of 
their  work  with  other  laboratories, 
program  managers  are  much  more 
likely  to  correct  non-complying 
performance.     Similarly,   they  are 
in  a  much  better  position  to  demand 
that  accreditors  take  action  to 
correct  deficiencies  indicated 
through  interlab  reference  specimen 
testing. 


Summation 

UL's  experiences  with  accredita- 
tion programs  have  not  been  entirely 
on  the  negative  side.  We  are  cur- 
rently participating  in  some  accred- 
itation programs  that  have  been  care- 
fully and  knowledgibly  conceived  and 
uniformly  and  equitably  implemented. 
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The  success  or  failure  of  any  lab- 
oratory accreditation  program  ulti- 
mately depends  upon  the  persistence 
and  judgment  of  the  administrators  to 
implement  the  program  in  a  uniform, 
equitable  manner  with  insistence  upon 
compliance  with  all  features  of  the 
program's  criteria. 

It  is  perhaps  appropriate  to 
close  with  a  quote  by  Elbert  Hubbard, 
who  wrote  the  famous  "Message  to 
Garcia."     In  his  published  notebook 
he  said:     "We  shall  never  get  the 
right  idea  of  work  until  we  see  at 
the  bottom  of  it  is  public  service." 
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Abstract 

In  Laboratory  Accreditation  Programs  by  Discipline  -  the  laborato- 
ries are  accredited  for  performance  of  groups  of  tests  or  specific  tests 
within  a  discipline  of  testing.     Individual  laboratories  may  be  accred- 
ited in  more  than  one  discipline  and  for  one  or  more  classes  within  each 
discipline.     The  evaluation  process  for  discipline  accreditation  covers 
the  technical  and  ethical  competence  of  the  personnel,  and  the  equipment 
and  quality  control  procedures  for  the  discipline/groups  of  tests  that 
are  enumerated  in  the  laboratory's  application  for  accreditation. 

Key  words:     Classes  of  tests;  directory  of  accredited  laboratories; 
discipline;  laboratory  accreditation. 


1.  Introduction 

The  principles  of  "accreditation  by  dis- 
ciplines" are  not  generally  understood,  al- 
though they  are  utilized  successfully  by  the 
largest  national  system,  Australia's  Na- 
tional Association  of  Testing  Authorities 
(NATA) .     They  are  also  utilized  by  the 
<\merican  Association  for  Laboratory  Accred- 
itation (AALA) ,  and  are  acceptable  to  the 
National  Voluntary  Laboratory  Accreditation 
Program  (NVLAP)  under  their  optional  pro- 
cedures (15  CFR  Part  7c)  for  private  sector 
Drganizations. 

2.     Nine  Technical  Disciplines 

The  NATA  and  AALA  systems  have  nine 
technical  disciplines  as  follows: 

1.  Acoustical  and  Vibration 

Measurement 

2.  Biological  Testing 

3.  Chemical  Testing 

4.  Electrical  Testing 

5.  Thermal  Testing 

6.  Mechanical  Testing 

7.  Metrology 


8.  Non-destructive  Testing 

9.  Optics  and  Photometry 

Each  discipline  has  sub-disciplines 
referred  to  by  NATA  as  sections  and/or 
classes  of  tests,  and  by  AALA  as  groups  of 
tests.     A  group  of  tests  within  the  elec- 
trical discipline  may  cover  a  type  of  test 
such  as  high  voltage  testing  or  be  by  broad 
product  classification  such  as  batteries. 

NATA  divides  the  Non-destructive  Test- 
ing discipline  into  seven  sections: 

Radiographic  examination  of  metals 
Radiographic  examination  of  non-metals 
Radiographic  examination  of  components 

and  assemblies 
Ultrasonic  examination  of  metals 
Ultrasonic  examination  of  non-metals 
Ultrasonic  examination  of  components 

and  assemblies 
Other  non-destructive  tests 

Each  section  is  divided  into  classes  of 
tests.     Applications  for  accreditation  may 
be  made  for  one  or  more  classes  of  tests  or 
for  one  or  more  items  within  a  class  of 
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tests.     Examples  of  the  breakdown  of  two  NATA 
sections  within  the  Non-destructive  Testing 
discipline  are  as  follows: 

Radiographic  Examination  of  Components 
and  Assemblies 

Aircraft  structures 
Components  and  assemblies 
(applicant  to  specify) 

Ultrasonic  Examination  of  Non-metals 
Ceramics  and  refractories 
Plastic  laminates 
Plywood 

Other  specified  non-metals 

The  above  examples  help  illustrate  that 
a  laboratory  is  evaluated  for  accreditation 
by  sub  and  sub-sub  disciplines,  not  for  all 
the  testing  within  the  discipline.     The  lab- 
oratory specifies  the  extent  of  its  capabil- 
ities in  its  application  and  requests  accred- 
itation accordingly. 

3.     Evaluation  Process  for  Discipline 
Accreditation 

All  valid  laboratory  accreditation  pro- 
grams involve  an  on-site  evaluation  of  the 
capabilities  at  the  laboratory.     After  the 
laboratory's  application  has  been  reviewed 
and  deemed  to  provide  adequate  information, 
trained,  professionally  qualified  inspectors 
are  assigned  to  visit  the  laboratory  to  eval- 
uate its  technical  competence  to  perform  the 
types  of  tests  indicated  in  the  application. 

The  evaluation  covers  the  personnel, 
equipment,  organization,  laboratory  and 
quality  control  procedures  for  the  discipline 
and  all  involved  types  of  tests.     Since  dis- 
cipline accreditation  is  more  broad-range 
than  accreditation  "by  product-by  standard" 
the  review  of  "personnel  and  adequacy  of  pro- 
cedures must  be  very  extensive.  Demonstra- 
tions of  typical  tests  during  initial  eval- 
uations and  follow-ups  are  required. 

The  initial  inspection  of  the  laboratory 
requires  a  minimum  of  one  day  and  up  to  sev- 
eral days  dependent  upon  the  size  of  the 
laboratory  and  the  number  of  classes  of  tests 
in  each  discipline.     A  similar  amount  of  time 
is  required  to  evaluate  the  same  laboratory 
for  a  single  product  being  tested  per  one  or 
a  limited  number  of  standards. 

4.     Discipline  Standards  and 
Accreditation  Criteria 

The  NATA  program  has  a  separate  booklet 
that  provides  information  on  requirements  for 
registration  in  each  discipline.     Each  book- 


let describes  the  coverage  of  the  disciplii 
and  the  classes  of  test,  the  application  a; 
evaluation  procedure  and  other  pertinent  i' 
formation. 

The  AALA  program  is  similar  and  each 
discipline  accreditation  program  is  based 
upon  voluntary  consensus  standards  that 
have  been  and  are  being  developed  in  ASTM 
and  other  standards  writing  organizations. 
These  separate  discipline  standards  follow 
the  generic  criteria  that  are  provided  in 
ASTM  Standard  E-548  "Standard  Practice 
Generic  Criteria  for  Use  in  the  Evaluation 
of  Testing  and/or  Inspection  Agencies"  and 
ISO  Guide  25  "Guidelines  for  Assessing  the 
Technical  Competence  of  Testing  Laboratori 

5.     Directory  of  Accredited  Laboratories 

An  important  function  of  all  laborato 
accreditation  programs  is  to  make  availabl 
to  all  interested  parties  on  a  current  bas 
a  directory  of  accredited  laboratories,  wi 
clear  designations  of  the  discipline  and 
types  of  tests  covered  by  the  accreditatio 
The  user  of  the  accreditation  program  must 
be  made  aware  of  the  basis  of  the  evalua- 
tions,  the  extent  and  limitations  of  the 
listings,  and  the  reevaluation  procedures. 
This  full  disclosure  is  important  since  th 
user  of  the  accreditation  and  testing  serv 
ices  must  determine  if  the  accreditation 
process  is  adequate  for  his  use  or  whether 
he  wants  to  add  additional  requirements  tc 
satisfy  his  specific  needs. 

In  addition  to  the  above  information, 
the  NATA  Register  of  Laboratories  details 
the  terms  of  the  accreditation  and  lists 
the  signatories.     The  terms  of  accreditatj 
are  those  classes  of  test  applicable  to  ti 
work  of  the  laboratory,  modified  by  rangef 
of  measurements  where  they  are  applicable, 
and  by  statements  of  uncertainty  of  measu: 
ment  where  they  are  applicable.     The  ap- 
proved signatories  are  the  laboratory  off; 
cers  to  whom  the  Council  has  given  specif: 
approval  to  sign  NATA  endorsed  test  docu- 
ments.    Endorsed  test  documents  may  be 
signed  by  these  signatories.     If  any  of  Ell 
leaves  the  laboratory,  he  is  automaticall; 
terminated.     The  NATA  Directory  is  in  the 
form  of  loose-leaf  sheets,  with  one  or  mo: 
sheets  for  each  laboratory.     Updating  she< s 
are  distributed  frequently. 

6.  Conclusion 

There  are  hundreds  of  laboratory  ac- 
creditation programs  covering  specific, 
limited  areas  of  test;  they  are  operated  j 
numerous  organizations,  such  as  corporati  s 
trade  associations  and  government  agencie 
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nd  are  generally  limited  to  their  own  use 
uch  as  for  a  product  certification  program, 
s  a  consequence  of  these  splintered  and  un- 
oordinated  accreditation  programs,  most 
aboratories  in  the  U.S.,  especially  the 
ulti-discipline  laboratories  doing  testing 
or  fees,  are  inspected  many  times  every  year 
his  becomes  expensive  and  represents  un- 
ecessary  duplication  of  efforts. 

The  only  national  systems  in  the  U.S., 
hat  have  as  their  objective  providing  ac- 
reditation  in  all  or  most  test  areas,  are 

:VLAP  (government  operated)  and  AALA  (private 

Rector  program). 


It  is  important  to  recognize  that  ac- 
reditation  by  discipline  is  a  broad-band 
pproach  and  as  such  permits  a  greater 
ortion  of  the  total  accreditation  assign- 
Lent  to  be  done  in  a  shorter  period  and  in 
more  efficient  manner. 
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The  paper  briefly  describes  the  constitution,  role  and  responsi- 
bilities of  the  Standards  Council  of  Canada,  which  is  the  accrediting 
authority  for  the  national  voluntary  accreditation  program  for  testing 
organizations  in  Canada.     It  also  touches  on  the  composition  of  the 
National  Standards  System  (NSS) ,  a  federation  of  independent  autonomous 
organizations  working  towards  the  further  development  and  improvement 
of  voluntary  standardization  in  the  national  interest.     This  system, 
which  is  under  the  aegis  of  the  Standards  Council,  will  include 
accredited  testing  organizations  as  one  component. 

The  document,  "Criteria  and  Procedures  for  Accreditation  of  Testing 
Organizations"  (CAN-P-4)  was  approved  and  published  by  Council  in  1978. 
A  limited  (pilot)  accreditation  program  is  being  conducted  this  year 
(1979)  in  order  to  evaluate  implementation  procedures  and  estimate  costs 
for  a  national  program.     It  is  hoped  that  this  limited  program  will 
provide  sufficient  information  for  Council  to  make  decisions  in  1980 
concerning  a  national  program. 


Key  words:  Accreditation;  Canada; 
Council  of  Canada;  Testing. 


1 .  Introduction 

This  presentation  describes  the 
Standards  Council's  accreditation  program 
for  Canadian  testing  organizations  but 
first  it  may  be  of  interest  to  provide  some 
information  on  the  constitution  of  the 
Council.     The  Standards  Council  of  Canada 
was  created  by  an  Act  of  Parliament  in  1970 
(The  Standards  Council  of  Canada  Act).  Its 
membership  consists  of  16  Federal  and 
Provincial  officials  and  41  representatives 
from  private  organizations  which  have 
interests  in  the  field  of  standardization. 
However,  it  is  important  to  emphasize  that 
the  Council  is  not  a  government  agency  nor 
are  its  employees  public  servants.  In 
effect,  it  is  independent  of  government  in 
its  policies  and  operation  although  it  is 
financed  by  a  grant  from  the  Federal 
Government . 
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The  prime  objective  of  the  Council 
(as  stated  in  the  Standards  Council  Act) 
is  to  foster  and  promote  voluntary 
standardization. 

The  Act  also  states  that  the  Counci 
in  carrying  out  its  objectives,  may 
"accredit,  in  accordance  with  criteria  a 
procedures  adopted  by  the  Council,  organ 
zations  in  Canada  engaged  in  standards 
formulation,  testing  and  certification 


The  Council  was  not  created  as  a  sujt 
organization  duplicating  existing  expertje 
and  facilities.  It  does  not  itself  prepfe 
write  or  publish  standards.  These  funct  ins 
are  performed  by  the  Standards -Writing 
Organizations  (SWOs)  who  are  accredited 
the  Council  in  accordance  with  approved 


criteria.    Similarly,  the  Council  does  not 
plan  to  certify  products  or  operate  testing 
laboratories  but  is  implementing  programs 
to  accredit  both  Certification  and  Testing 
Organizations . 

2 .    National  Standards  System  (NSS) 

The  National  Standards  System  is 
formally  defined  as  "A  federation  whose 
components  are  accredited  standards-writing, 
certification  and  testing  organizations, 
the  Canadian  committees  concerned  with 
international  standardization  and  the 
Standards  Council  of  Canada.     The  System 
provides  a  coordinated  approach  to  the 
development  and  advancement  of  voluntary 
standardization  in  the  national  interest". 

The  programs  for  accreditation  of 
certification  and  testing  organizations  are 
now  in  the  implementation  phase  and  the 
current  status  of  the  testing  program  will 
be  described  later. 

3.    SCC  -  Organization 

And  now  before  introducing  the  main 
topic  of  my  presentation,  I  feel  it  might 
be  of  some  value  to  briefly  describe  the 
SCC  organization.     The  Council  itself  meets 
only  twice  a  year  and  the  responsibility 
for  its  affairs  between  these  meetings  is 
delegated  to  a  nine  member  Executive 
Committee.    This  Committee  meets  six  times 
a  year. 

The  day-to-day  work  of  the  Council  is 
planned  and  co-ordinated  by  a  permanent 
staff  of  approximately  sixty.     Four  branches 
headed  by  Directors  are  responsible  to  the 
Executive  Director.     Three  of  these  branches 
are  located  at  the  Council  Headquarters  in 
Ottawa  with  one  (the  International  Stan- 
dardization Branch)  situated  in  Mississauga, 
Ontario  (near  Toronto) . 
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The  specialized  professional  expertise 
and  experience  is  provided  to  Council  by 
various  Advisory  Committees.     Thus  the 
Advisory  Committee  on  Certification  and 
Testing  (ACCT)  is  the  body  responsible  for 
supplying  advice  and  guidance  to  Council  on 
these  activities.     The  ACCT  is  a  24  member 
body  which  was  formed  in  1974  and  since  that 
time  has  operated  under  the  chairmanship  of 
Mr.  J.E.  Elliot,  Director  of  Engineering, 
a ^Vehicle  Quality  and  Safety,  Chrysler  Canada, 
-:*'-;Ltd. 
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4.    Accreditation  of  Testing  Organizations 

The  document  "Criteria  and  Procedures 
for  Accreditation  of  Testing  Organizations" 
(CAN-P-4)  was  approved  at  the  Twenty-First 
Council  Meeting  on  June  5,  1978.  Before 
discussing  this  document  a  resume  of  the 
background  of  the  program  including  a  brief 
outline  of  its  scope  might  be  in  order. 

One  point  which  I  would  like  to  stress 
is  that  the  Council  program  for  the  accre- 
ditation of  testing  organizations  envisages 
testing  as  a  function  in  itself  and  not 
solely  in  its  relationship  to  certification 
programs.    The  goal  is  to  develop  a  volun- 
tary accreditation  program  that  will 
identify  those  organizations  that  are  com- 
petent in  their  fields  of  testing  and  also 
promote  a  general  improvement  of  testing 
services  in  Canada.     Such  a  program  would 
permit  competent  organizations  in  all  fields 
of  testing  to  qualify  for  national  accre- 
ditation.    This  would  include  calibration 
services,  testing  for  product  development, 
research  and  contract  monitoring  as  well 
as  testing  in  support  of  certification 
programs . 

It  is  planned  to  provide  an  accredita- 
tion program  in  support  of  product  and 
technical  testing  in  all  fields.  However, 
it  should  be  noted  that  the  National 
Research  Council  of  Canada  is  responsible 
for  maintaining  Canada's  national  metro- 
logical  standards  while  the  Department  of 
Consumer  and  Corporate  Affairs  is  respons- 
ible for  regulating  and  inspecting 
measuring  devices  used  in  trade. 

Several  other  Federal  Government 
departments  conduct  programs  to  determine 
if  manufacturing  and  service  organizations 
are  qualified  in  accordance  with  legislated 
requirements  or,  in  other  cases,  to 
specific  contract  specifications. 

In  addition  to  these  government 
activities  there  are  also  some  non- 
government agencies  operating  qualification 
programs.     One  example  is  the  standard 
"Qualification  Code  for  Concrete  Testing 
Laboratories"  (CSA  A283-1974) .  This 
standard  was  developed  by  the  Canadian 
Standards  Association  which  is  an  accre- 
dited standards-writing  organization  of 
the  National  Standards  System.     The  program 
is  operated  by  the  Certification  branch  of 
that  organization. 
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Reference  was  made  above  to  the  appro- 
val by  Council  of  a  criteria  and  procedure 
policy  document  for  the  accreditation  of 
testing  organizations.    This  document  was 
published  in  September,  1978,  with  the 
title  "CAN-P-4  -  Criteria  and  Procedures 
for  Accreditation  of  Testing  Organizations". 

In  addition  to  approving  the  draft 
CAN-P-4,  Council  also  authorized  the 
Advisory  Committee  on  Certification  and 
Testing  (ACCT)  to  proceed  with  the  develop- 
ment of  implementation  procedures,  the 
investigation  of  legal  aspects  and  the 
implementation  of  a  limited  program  to 
estimate  costs.    The  first  two  activities 
have  been  completed  and  the  implementation 
of  a  limited  program  has  been  initiated. 
This  limited  program  is  essentially  a  pilot 
project  and  on  its  completion  (the  evalua- 
tion of  a  small  number  of  representative 
testing  organizations)  recommendations  will 
be  made  to  Council  concerning  the  imple- 
mentation of  a  national  accreditation 
program. 

In  format  CAN-P-4  consists  of  a 
Foreword,  Preface,  nine  Sections  and  three 
Annexes . 

There  are  seven  criteria  which  are 
summarized  below: 

Criterion  1:    Ability  to  operate  and 
maintain  an  adequate  testing  capability. 

Criterion  2:  Staff-administrative 
competence. 

Criterion  3: 


Staff-technical  com- 


petence . 

Criterion  4: 


Adequate  facilities, 


Criterion  5:     Documented  and  accept- 
able procedures  for  maintenance  of  records. 

Criterion  6:     Willingness  to  allow 
examination  of  records,  etc. 

Criterion  7:     Independence  of 
operation. 

CAN-P-4  is,  in  essence,  a  "generic" 
type  document  in  that  its  criteria  and 
requirements  must  be  met  by  any  applicant 
for  accreditation  regardless  of  the  field, 
or  fields,  of  testing  in  which  it  is 
engaged.     "Field  of  Testing"  is  defined  as 
"...a  range  of  related  testing  activities 


as  defined  by  the  Standards  Council  of 
Canada"  and  in  Annex  "C"  to  CAN-P-4  they  are 
listed  as  -  Chemical,  Electrical,  Mechanical 
Non-Destructive  and  Physical. 

4.2    Implementation  of  the  Accreditation 
Program 

Accreditation  of  testing  organizations 
is  a  two-level  procedure.     The  first  level 
consists  of  compliance  with  CAN-P-4  and  the 
second  level  is  defined  by  supplementary 
documents  related  to  that  portion  of  the 
specific  field  for  which  the  applicant  has 
requested  accreditation.     These  supple- 
mentary documents  may  be  National  Standards 
of  Canada,  other  established  standards  or  a 
client's  specifications.     It  will  be  the 
responsibility  of  the  applicant  to  supply 
this  documentation  and  identify  those 
portions  of  the  field  (or  fields)  of  testing 
for  which  it  claims  competence. 

It  is  realised  that  while  a  small 
testing  organization  would  probably  be 
interested  in  being  accredited  for  a  small 
portion  of  a  specific  field,  a  large  organi 
zation  might  well  request  accreditation  in 
several  fields. 

The  evaluation  of  a  testing  organiza- 
tion is  conducted  by  a  Testing  Accreditatior 
Sub-Committee  (TASC)  of  the  Advisory 
Committee  on  Certification  and  Testing 
(ACCT).     This  sub-committee  consists  of  a 
chairman  and  two  members  and  it  is 
authorized  to  request  assistance  from 
specialists  when  necessary.    After  the 
evaluation  is  completed  the  TASC  makes  its 
recommendation  through  ACCT  to  Council. 
The  Standards  Council  is  the  approving 
authority  for  accreditation. 

5 .     Other  National  Programs 

There  has  been  a  considerable  amount 
of  documentation  generated  in  the  past  few 
years  by  other  countries  regarding  labora- 
tory accreditation. 

Some  national  accreditation  programs 
are  already  in  operation  and  the  Standards 
Council  has  been  able  to  take  advantage  of 
the  previous  effort  which  has  been  expended 
by  others  in  this  area. 

Specifically,  I  would  like  to  refer  to 
the  work  done  by  the  U.S.  Department  of 
Commerce  and  the  National  Bureau  of  Stan- 
dards in  the  development  of  their  NVLAP 
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program.     Other  sources  from  the  U.S.  are 
ASTM,  which  produced  its  Standard  Recom- 
mended Practice  E-548-76,  and  the  American 
Council  of  Independent  Laboratories  (ACIL) 
which  has  also  been  active  in  this  area. 
The  program  of  the  American  Association 
for  Laboratory  Accreditation  is  also  being 
noted  with  interest. 

From  the  U.K.  we  have  also  benefited 
by  the  work  done  by  the  British  Standards 
Institution  (BSI)  and  the  British  Cali- 
bration Service  (BCS) .     Their  procedures 
and  documentation  have  been  of  considerable 
assistance  to  us. 

We  have  drawn  on  the  Australian 
experience  where  the  National  Association 
of  Testing  Authorities  (NATA)  has  had  a 
laboratory  registration  program  in  exis- 
tence for  over  thirty  years  -  indeed  it  is 
probably  the  pioneer  in  this  respect.  A 
more  recent  national  scheme  is  that  of  New 
Zealand  where  the  Testing  Laboratory 
Registration  Council  (TELARC)  has  been  in 
operation  for  the  past  few  years. 

On  the  international  scene  the 
Council  has  participated  in  the  1977  and 
1978  International  Conferences  on  Recog- 
nition of  National  Programs  for  Testing 
Laboratories  (ILAC)  and  will  be  repre- 
sented at  ILAC  1979. 

6.  Conclusion 

It  is  anticipated  that  the  program 
for  accreditation  of  testing  organizations 
will  assist  in  identifying  those  organiza- 
tions which  have  demonstrated  competence. 
Objectivity,  impartiality  and  accuracy  are 
the  major  elements  to  be  considered  in 
establishing  the  credibility  of  an  organi- 
zation which  provides  such  testing 
services . 

A  Directory  of  organizations  which 
have  demonstrated  their  ability  to  comply 
on  a  continuing  basis  with  common  agreed 
criteria  and  procedures  will,  it  is 
believed,  considerably  enhance  their 
'national  status  and  recognition.     It  also 
should  be  of  considerable  value  to 
potential  users  of  their  services. 

The  program  for  accreditation  of 
testing  organizations  is  a  part  of  the 
National  Standards  System  which  is  by 
definition  a  voluntary  federation  of 
organizations.    The  success  of  this  pro- 
gram will  depend  to  a  large  extent  on 
the  response  it  receives  from  both  the 


participating  organizations  and  the  users 
of  their  services.     It  is  still  too  early 
to  determine  this  response  or  to  assess 
the  impact  on  national  testing  activities. 
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National  Bureau  of  Standards  Special  Publication  591,  Proceedings  of  the  National  Conferenc 
on  Testing  Laboratory  Performance:  Evaluation  and  Accreditation,  held  at  NBS ,  Gaithersburg, 
MD,  September  25-26,  1979.        Issued  August  1980 


ECONOMIC  EFFECT  OF  LABORATORY  ACCREDITATION 

DAVID  KRASHES 

PRESIDENT,   MASSACHUSETTS  MATERIALS  RESEARCH,  INC. 
WEST  BOYLSTON,   MASS.  01583 

A  fear  of  some  commercial  laboratory  owners  about 
national  accreditation  is  that  laboratories  remaining 
unaccredited  will  be  able  to  charge  lower  prices  and, 
thus,  garner  the  majority  of  the  testing  business.  This 
study  determined  average  prices  for  tensile  tests  of 
metals,  relative  amounts  of  these  tensile  tests  performed 
in  "well-accredited"  and  "poorly-accredited"  laboratories; 
and  determined  the  effect  of  being  accredited,  under 
today's  systems  of  accreditation,  on  the  quantities  of 
tensile  testing  business  obtained  by  commercial  labora- 
tories.    In  geographic  regions  where  being  "accredited" 
is  recognized  as  important  by  a  major  segment  of  manu- 
facturing industry,  most  of  the  tensile  testing,  regardless 
of  price,   is  performed  in  "well-accredited"  laboratories. 
In  geographic  regions  where  no  strong  segment  of  consuming 
industry  requires  "accreditation",  most  of  the  tensile 
testing  is  done  in  laboratories   "poorly  accredited"  by 
today's  systems.     Thus,   if  the  consuming  industry  recog- 
nizes the  value  of  accreditation,   there  will  be  an  economic 
advantage  for  accredited  laboratories:     they  will  be  able 
to  charge  more  and  also  may  gain  more  testing  business. 
In  regions  where  no  strong  segment  of  industry  recognizes 
the  value  of  accreditation,  accredited  laboratories  will 
suffer  economically.     The  economic  effect  of  any  proposed 
laboratory  accreditation  program  should  be  studied  prior 
to  initiating  the  program. 

Key  words:     Accreditation;   costs;   economic;  prices;  testing. 

1.     Introduction  tory  owners  seem  to  say  they  spend 

great  quantities  of  time  preparing 

The  owner  or  manager  of  a  commer-  documentation  and  undergoing  inspec- 
cial  laboratory  is  always  prey  to  the     tions.     Presumably,  there  would  be  ai 
suspicion  that  competitors  charging         economic  effect  of  national  accredi-: 
lower  prices  will  garner  most  of  the  tation. 
available  business.     Many  types  of 

testing  are  looked  upon  by  consuming  The  Federal  Register  of  April  2 

industries  and  governmental  agencies  1979  published  a  comment  by  the  Depa 
as  commodities:     quality  is  either  ment  of  Commerce,  page  24276,  that 

taken  for  granted  or  not  cared  about.     said  quite  strongly  that  there  is  noj 

need  to  study  the  economic  impact  of 
Everybody  says  laboratory  ac-  accreditation  programs,  because  thos 

creditation  costs  money.     All  labora-     requesting  the  accreditation  program 
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will  already  have  considered  the  eco- 
nomic impact.     My  interest  was  piqued. 
I  had  neither  heard  nor  read  of  factu- 
al economic  studies. 

I  proposed  to  Dr.  Berman  that  the 
economic  impact  of  accreditation  on 
tensile  testing  ought  to  be  studied, 
and  I  proposed  to  do  so  on  a  large 
number  of  laboratories.     I  suggested 
that  I  would  come  here  today  to  report 
on  data  and  results  of  which  I  had, 
at  that  time,   no  knowledge.     I  see, 
as  a  matter  of  fact,  that  I  am  the 
only  speaker  in  the  entire  program 
discussing  economics. 

2.     The  Study 

The  tensile  test  of  a  metal  is 
a  pull  in  a  testing  machine  of  a 
sample  that  has  already  been  prepared 
to  a  specific  shape  and  size.  Pre- 
paration charges  for  test  samples 
vary  and  are  often  quoted  as  a  fixed 
price.     However,   the  price  of  the 
test  alone  is  almost  always  quoted 
on  a  fixed  price  basis. 

Within  certain  bounds,   the  test 
must  be  performed  approximately  the 
same  way  by  almost  all  laboratories. 
Properly  done,   about  5  to  7  tensile 
tests  per  hour  are  possible.  Short- 
cuts, which  sometimes  lessen  quality, 
enable  as  many  as  10  to  12  tests  per 
hour  to  be  done.     This  lowers  cost 
and  can  lower  price.     The  price  any 
one  commercial  laboratory  charges  for 
a  tensile  test  can  be  quoted;  is 
likely  to  reflect  the  way  the  labora- 
tory does  the  test;   and  also  may  re- 
flect competitive  conditions  the 
laboratory  faces. 

I  sent  questionnaires  to  over 
100  commercial  laboratories  in  the 
eastern  United  States  and  asked  the 
prices  they  charged  for  tensile  tests. 
I  also  asked  questions  which  would 
indicate  whether  or  not  each  labora- 
tory was  in  a  condition  that  would 
probably  enable  it  to  pass  an  ac- 
creditation inspection.     A  copy  of 
the  questionnaire  is  attached  to  the 
written  copy  of  this  talk.     As  you 
might  expect  of  a  novice,   I  didn't 
ask  exactly  the  right  questions;  and 
I  had  to  telephone  a  great  many  lab- 
oratories both  to  get  answers  and  to 
form  opinions  as  to  their  accredita- 
tion status. 


I  divided  "accreditation  status" 
into  2  categories:     "good"  and  "fair". 
Laboratories  that  had  quality-control 
manuals  and  had  already  been  approved 
by  accreditation  systems  that  I  be- 
lieved to  be  thorough   (such  as  the 
aircraft  engine  industry) ,   I  consid- 
ered to  be  in  the  "good"  category. 
Laboratories  without  quality-control 
manuals  or  which  had  been  inspected 
only  by  customers  such  as  welding 
fabricators  in  the  nuclear  industry, 
who  often  perform  "less-than-rigorous" 
inspections,   I  considered  to  be  in 
the  "fair"  category.     I  have  person- 
ally visited  a  number  of  the  labora- 
tories in  this  study  and  I  talked  by 
telephone  to  many  more.     While  there 
may  be  disagreement  with  my  definition 
of  accreditation  categories,   I  am 
quite  certain  that  I  placed  most  of 
the  laboratories  in  the  proper  cate- 
gory . 

3 .     The  Data 

In  the  eastern  United  States, 
with  perhaps  not  enough  laboratories 
reporting,   Table  1  shows  the  average 
price  overall  for  tensile  tests,  and 
the  average  prices  charged  by  "fair" 


Table  1.     Average  Price  Per  Test 

Eastern  U.S. 
(Incomplete  Data-41  Laboratories] 

Overall  $12.12 
22  Accredited  "Good"  11.86 
16  Accredited  "Fair"  13.53 


and  "good"  categories  of  laboratories 
The  "good"  and  "fair"  are  within  12% 
and  2%,   respectively,  of  the  overall 
average.     All  prices  over  $25.  and 
all  laboratories  doing  under  5  tests 
per  week  are  excluded. 

Now,  here  is  a  rule  of  thumb 
regarding  pricing:     an  industrial 
customer,  who  has  been  regularly 
using  a  certain  vendor,  will  not 
shift  to  another  vendor  on  a  price 
basis  alone  if  the  reduction  of 
price  is  less  than  10%.     In  the 
laboratory  industry,  users  seem  to 
require  even  greater  reductions  of 
price  to  induce  them  to  change  from 
one  commercial  laboratory  to  another. 
The  data  of  Table  1  doesn't  show 
enough  variation  of  price  to  cause 
shifts  of  customers.     Oddly,  the 
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"good"  category  seems  to  charge  a 
lower  average  price. 

Table  2,   in  which  the  volume  of 
tests  performed  in  various  labora- 
tories is  considered,  throws  a 
different  light  on  the  subject.  Here 
we  see  that  most  of  the  tensile  tests 
in  the  eastern  United  States  are  per- 
formed by  the  "good"  category  at 


Table  2.     Price  and  Quantity 
Eastern  U.S. 


charged  by  the  "good"  laboratories 
is  than  the  overall  average.  The 
"fair"  laboratories  are  almost  at 
the  average.     This  is  a  situation  in 
which  price  differentials  might  in- 
duce movements  of  customers  from 
higher  priced  to  lower  priced  labora- 
tories . 

So,  what  do  we  find?     Table  4 
shows  that  most  of  the  tensile  test- 
ing in  New  England  is  performed  at 
a  price  above  the  New  England  average 


Tests 
Per  Day 

20  or  more 
10-20 
5-10 
Under  5 


Average 
Price 

$11.58 
8.78 
12.45 
13.70 


Where 
Done 

100%  in  Good 

60%  in  Good 

60%  in  Good 

30%  in  Good 


Table  4 .     Price  and  Quantity 
New  England 


prices  lower  than  the  eastern  average. 
Although  not  shown  in  the  table,  2/3 
of  all  tests  done  in  the  eastern 
United  States  are  priced  below  average 
and  half  of  all  the  tests  done  are 
performed  in  the  "good"  category  lab- 
oratories . 

These  broad-scale  geographic  da- 
ta must  be  considered  with  caution. 
They  cover  too  much  territory  geo- 
graphically, whereas  most  tensile 
testing  of  metals  is  done  on  a  region- 
al basis.     Probably  most  commercial 
laboratories  are  regional:  serving 
a  small  geographic  region.     In  the 
eastern  United  States,  probably  very 
few  tensile  test  samples  travel  more 
than  100-150  miles.     The  foregoing 
data  overlooks  regional  competition, 
and  the  data  to  follow  provides  sub- 
stantiation . 

In  New  England,   Table  3,  there 
are  5  "good"  and  6   "fair"  laboratories 


Table  3.     Average  Price  Per  Test 
New  England-11  Laboratories 


Overall  Eastern 
Overall  N.E. 

5  "Good" 

6  "Fair" 


$12.12 
7.75 
11.50  (+48%) 
7.92  (-2%) 


performing  tensile  tests.     See  how 
much  higher,   48%,   the  average  price 


Tests 
Per  Day 

20  or  more 
10-20 
5-10 
under  5 


Average 
Price 

$  9.85 
10.40 

5.50 


Where 
Done 

100%  in  Good 
60%  in  Good 

0%  in  Good 


of  $7.7  5  and  most  of  it  is  performed 
in  "good"  category  laboratories.  In 
fact,   60%  of  all  the  tensile  testing 
is  performed  in  the  "good"  labora- 
tories at  above  the  overall  average 
price . 

In  New  England,  most  of  the 
purchasers  of  tensile  testing  are 
willing  to  pay  higher  prices  in  or- 
der to  have  their  tests  performed 
by  "accredited"  laboratories. 
New  England,   incidentally,  is  the 
headquarters  of  two  of  the  nation's  ' 
largest  aircraft  engine  manufacturer 
Pratt  &  Whitney  and  General  Electric' 
Both  companies  have  very  strong  lab- 
oratory inspection  and  accreditation 
systems.     Subcontract  vendors  to 
these  two  companies  abound,  and  they 
are  forced  to  use  accredited  labora 
tories  for  their  testing.     Many  othe 
industries  in  New  England  recognize 
that  accreditation  by  Pratt  &  WhitneJ 
and  General  Electric  means  a  labora- 
tory does  good  work. 

For  the  Metropolitan  New  York 
City  area,  which  includes  New  York 
City,  Long  Island,  and  northeastern 
New  Jersey,  Table  5  shows  the  averag 
price  charged  by  "good"  category 
laboratories  to  be  higher  than  the 
overall  average;  while  the  average 
price  charged  by  "fair"  laboratories 
is  significantly  lower  than  the 
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overall  average.     Here,  again,  the 
price  discrepancy  seems  great  enough 
to  warrant  purchasers  moving  from  the 
higher  priced  to  the  lower  priced 
laboratories.     Yet,  Table  6,  we  find 
that  all  laboratories  doing  more  than 
10  tests  per  day  are  in  the  "good" 
category.     In  fact,   70%  or  more  of 
all  the  tensile  tests  done  in  this 
region  are  at  above  the  average  price 
for  the  region  and  are  performed  in 
"good"  category  laboratories. 

Table  5.     Average  Price  Per  Test 
Greater  Metropolitan  New  York 
7  Laboratories 


Overall  National 
Overall 
5  "Good" 
2  "Fair" 


$12.12 
13.86 

15.50  (+12%) 
9.75  (-30%) 


In  another  small  geographic  are 
near  New  York,  of  five  laboratories 
performing  tensile  tests,  two  are 
"good".     Complete  data  on  volume  of 
tests  was  not  obtained,  but  one 
"good"  laboratory,  charging  a  price 
60%  of  that  charged  by  the  others, 
was  doing  the  heaviest  volume  of 
tensile  testing  found  in  the  eastern 
United  States. 

This  seems  to  indicate  that  a 
well-accredited  laboratory  that  is 
able  to  charge  a  lower  price  can 
attract  a  very  significant  volume  of 
the  tensile  testing  market. 

The  Philadelphia  area,  Tables  7 
and  8 ,  shows  a  complete  reversal  of 
anything  discussed  so  far.  Here, 


Table  6.     Price  and  Ouantity 
New  York  Area 


Tests 
Per  Day 

20  or  more 
10-20 
5-10 


Average 
Price 

$15.00 
15.00 
12.33 


Where 
Done 

100%  in  Good 
100%  in  Good 
33%  in  Good 


The  Metropolitan  New  York  region 
is  complex  and  less  well  understood. 
There  is  a  significant  aircraft  in- 
dustry accreditation  program,  and 
there  may  be  a  "rub-off"  factor. 
There  may  also  be  a  transportation 
problem:     users  hesitating  to  entrust 
samples  to  a  slow  and  difficult 
shipping  situation,   and,  hence,  using 
the  nearest  laboratory. 

Within  the  region,  there  is  one 
small  geographic  area  in  which  two 
laboratories  are  "good"  and  one  lab- 
oratory is  "fair".     One  "good"  labor- 
atory and  the  "fair"  laboratory  com- 
pete "head-on"  at  the  same  low  price. 
The  "good"  laboratory  does  about  50% 
more  tests  than  the  other.  Surpris- 
ingly,  the  third  laboratory,  which  is 
also  "good",   charges  70%  more  than 
the  other  two  and  still  obtains  30%- 
50%  of  the  tensile  testing  business 
in  the  area.     This  seems  to  indicate 
that  purchasers  of  tensile  testing 
in  this  sub-area  value  accreditation 
more  than  price. 


Table  7.     Average  Price  Per  Test 
Greater  Philadelphia  Area 


Overall  National 
Overall 
3  "good" 
3  "fair" 


$12.12 
10.33 

15.00  (+45%) 
5.70  (-45%) 


Table  8 .     Price  and  Quantity 
Philadelphia  Area 


Tests 
Per  Day 


Average  Where 
Price  Done 


20  or  more 
10-20 
5-10 

less  than  5       12.25     75%  in  good 


6.50       0%  in  good 


there  is  a  severe  price  discrepancy 
between  the  "good"  and  "fair"  cate- 
gories.    The  price  discrepancy 
appears  great  enough  to  make  users 
of  tensile  testing  move  to  the  lower 
priced  laboratories.     The  "good" 
category  laboratories  have  only  25% 
of  the  tensile  testing  market  in  the 
region  that  extends  from  Baltimore 
to  central  New  Jersey  and  westward 
halfway  across  Pennsylvania. 

This  region  contains  a  heavy 
foundry  industry,  diverse  mundane 
manufacturing  industries,  and  no 
strong  aircraft  or  nuclear  industry 
that  relies  heavily  on  independent 
testing  laboratories.     This  is  a 
region  in  which  a  tensile  test  is  a 
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commodity.     Price,   therefore,  is  a 
very  strong  consideration.  Published 
advertisements  and  visits  to  labora- 
tories indicate  that  sub-standard 
techniques  may  be  in  use  and,  thus, 
may  hold  down  cost. 

4 .  Conclusion 

There  is  an  economic  effect  of 
laboratory  accreditation.     In  geo- 
graphic regions  where  most  consumers 
of  tensile  testing  believe  accredi- 
tation means  quality,   the  laboratories 
that  seem  to  be  best  qualified  are 
getting  the  major  share  of  the  busi- 
ness.    This  is  so  regardless  of 
whether  they  charge  higher  or  lower 
prices.     In  geographic  regions  where 
consumers  of  tensile  testing  are  not 
aware,  on  a  widespread  basis,  of 
accreditations  and  possible  variations 
of  quality,   the  laboratories  charging 
the  lowest  prices  are  getting  most 
of  the  business. 

Thus,  if  industry  is  aware  of 
the  value  of  laboratory  accreditation, 
the  economic  effect  of  national  ac- 
creditation will  be  good  for  accredi- 
ted commercial  laboratories:  they 
will  obtain  more  business  and  at 
higher  prices.     If  industry  does  not 
care  about  laboratory  accreditations 
and  quality,  the  lion's  share  of  the 
business  will  go  to  lower-priced 
laboratories,  whether  they  are  ac- 
credited or  not. 

Organizations  desiring  a  national 
laboratory  accreditation  procedure 
for  a  given  test  or  category  of  tests 
must  study  the  potential  economic 
effect.     This  should  be  done  by 
gathering  data  on  a  regional  basis  in 
order  to  avoid  disruptions,  "split- 
ting the  market"  for  the  test,  in 
some  regions. 

A  "split"  market"  would  mean 
that  some  commercial  laboratories  in 
a  region  would  forego  either  the 
accredited  or  the  unaccredited  por- 
tion of  their  business.     The  effect 
would  be  to  lessen  the  capacity  for 
doing  that  type  of  test  in  the  region. 
Since  commercial  laboratories  are  an 
integral  part  of  much  manufacturing 
industry,  a  decrease  of  testing 
capacity  could  be  disruptive  to 
industry . 


The  economic  effect  of  a  labora- 
tory accreditation  program  cannot  be 
taken  for  granted.     The  potential 
economic  effect  must  be  studied 
regionally  to  be  sure  that  requester; 
of  a  LAP  from  one  region  do  not  dis- 
rupt the  market  for  testing  in  anoth< 
region . 

5.     Appendix  -  Questionnaire 
Sent  to  Laboratories 

What  do  you  charge  for  perform- 
ing a  tensile  test  (not  including 
machining)  of  a  standard  metal  sampL 

Do  you  perform  your  tests  or 
keep  your  records  in  accord  with  re- 
quirements of  any  industry,  govern- 
ment agency,  company  or  anyone  (do 
not  include  ASTM) .      (If  you  follow 
only  ASTM,  your  answer  should  be 
"no" . ) 

IF  YES 


Are  you  regularly  (at  least 
once  each  year)  inspected  by  the 
accrediting  agency? 

If  yes,  do  they  give  you  a 
certificate  or  place  you  on  an 
"approved  list"? 

If  yes,  how  many  different 
accreditations  like  the  above  do 
you  undergo  for  tensile  testing? 

Do  you  have  a  typed  or  written 
manual  that  describes  the  following: 
how  you  perform  a  tensile  test,  how 
you  prepare  and  maintain  test  re- 
cords, how  you  check  the  quality  of 
your  tensile  tests,  who  in  your  lab- 
oratory is  qualified  to  perform 
tensile  tests? 
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THE  ROLE  OF  THE  QUALITY  CONTROL  MANUAL 
IN  THE  INSPECTION  AND  TESTING  LABORATORY 

ROBERT  J.  WENING 

B0WSER-M0RNER  TESTING  LABORATORIES ,  INC. 
P.  0.  BOX  51 
DAYTON,  OHIO  45401 

An  indispensable  tool  for  every  testing  laboratory  is  an  ade- 
quate quality  control  manual.     Such  a  document  is  a  cornerstone  of 
the  laboratory's  credibility  and  an  important  ingredient  in  its  as- 
sessment for  accreditation. 

Several  years  ago,  the  American  Council  of  Independent  Labora- 
tories (ACIL) ,  an  organization  with  a  long  history  of  interest  in 
laboratory  accreditation,  made  a  significant  contribution  in  this 
area.     ACIL  commissioned  its  Laboratory  Accreditation  Committee  to 
develop  a  guidebook  on  laboratory  quality  control.     The  committee's 
product  was  published  by  ACIL  in  19  76;  its  title  is  "Manual  of  Prac- 
tice:    Quality  Control  System:     Requirements  for  a  Testing  and  In- 
spection Laboratory." 

In  this  paper,  ACIL  will  review  the  development  of  the  manual 
and  its  contents.     Included  in  it  are  sections  on  Organization; 
Operational  Procedures;  Personnel;  Equipment  and  Calibration;  Refer- 
ence Samples;  Recommended  Personnel  Basic  Requirements;  Inventory  of 
Standards  and  Equipment  Requiring  Calibration;  and  Sources  of  Refer- 
ence Samples. 

The  paper  will  also  assess  the  impact  of  this  manual  on  the  lab- 
oratory community.     Samples  of  adaptations  of  the  manual's  guidelines 
by  individual  laboratories  in  various  disciplines  will  be  presented. 
Also,  reference  to  the  manual  by  other  bodies,  such  as  Government 
agencies,  will  be  discussed. 

Finally,  the  paper  will  estimate  the  importance  of  quality  con- 
trol manuals  and  programs  in  the  day-to-day  operation  of  a  laboratory 
and  in  relation  to  the  broader  issues  of  evaluating  and  accrediting 
testing  laboratories. 

Key  words:     Calibration;  equipment;  evaluation;  laboratory;  manual; 
personnel;  quality  assurance;  quality  control;  systems. 
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Dating  back  to  the  sixties  and  early 
seventies  the  independent  laboratories, 
especially  those  involved  with  the  aero- 
space industry,  were  in  quite  a  dilemma. 
(Although  the  term  "independent"  is  used 
in  this  paper  when  referring  to  labora- 
tories, the  principles  of  Quality  Control 
apply  to  all  types  of  laboratories.)  The 
laboratories  began  to  feel  the  pinch  when 
multitudinous  clients  and  governmental 
agencies  began  their  own  methods  and  inter- 
pretations of  Quality  Control  Programs  and 
audits  of  testing  laboratories.     The  lab- 
oratories also  began  to  get  requests  for 
"Who  certifies  you?"  or  "Who  accredits 
you?"  or  "What  national  agency  qualifies 
you?".     Some  laboratories  were  hiring 
people  strictly  for  the  purpose  of  working 
with  auditors.     It  was  at  this  point  near- 
ly a  decade  ago  that  a  group  of  us  in  the 
American  Council  of  Independent  Laborator- 
ies decided  that  something  must  be  done  to 
develop  a  uniformity  in  the  system.  When 
the  concept  of  a  laboratory  qualification 
or  accreditation  program  was  presented, 
naturally  a  committee  was  formed  to  develop 
the  idea.     Many  people  in  the  laboratory 
business  said  that  it  was  impossible  to 
accredit  laboratories  or  to  develop  cri- 
teria due  to  the  great  diversity  of  lab- 
oratories and  types  of  testing  and  inspec- 
tion.    However,   in  developing  the  concept, 
it  became  so  simple  that  it  seemed  ridic- 
ulous that  there  are,   in  essence,  just 
three  main  ingredients  in  an  accreditation 
program  for  a  laboratory,  or  really  any 
type  of  service.     They  are:     People,  Equip- 
ment and  a  Quality  Control  Program.     It  is 
not  necessary  to  accredit  to  every  specifi- 
cation, or  to  every  material,  or  even  to 
every  product.     This  is  the  muddle  in  which 
we  have  been  involved.     As  we  progressed, 
it  became  more  clear  that  the  backbone  of 
any  accreditation  program  was  to  be  the 
Quality  Control  Aspect. 

It  soon  became  quite  apparent  that  in 
order  to  have  an  accreditation  program,  it 
was  necessary  to  have  an  "accreditor"  and 
"accreditees".     The  accreditees  we  had, 
they  were  the  members  of  ACIL  but  the  ac- 
creditor was  lacking.     If  we,   the  ACIL, 
were  to  be  the  accreditation  agency,  then 
in  essence,  it  would  be  a  self-certifica- 
tion program  and  lack  the  meaning  that  was 
wanted  and  needed. 

The  emphasis  was  then  switched,  by  the 
committee  and  council,  to  attempt  to  devel- 
op a  Quality  Assurance  System  that  would  be 
uniform  for  testing  laboratories  regardless 
of  field  of  operation  and  be  readily  adapt- 
able for  an  accreditation  system. 


The  Quality  Assurance  System 

The  Quality  Assurance  System  was  then 
examined  and  the  keys  to  the  concept  were 
established. 

These  keys  were: 

Laboratory  Organization 
Operational  Procedures 
Personnel 

Equipment  and  Calibration 
Reference  Samples 

Let  us  examine  each  of  these  keys  in 
some  detail: 

Laboratory  Organization 

A  look  at  the  Laboratory  Organization 
provides  an  excellent  perspective  of  the 
Quality  Assurance  Program.     It  gives:  the 
type  of  work  that  can  be  performed,  the 
geography  covered,  the  function  of  the  main 
laboratory  and  branches,  historical  data, 
structure  or  organization  for  capabilities, 
a  look  at  the  operational  and  support  de- 
partments, and  indication  of  services 
available . 

Operational  Procedures 

Operational  Procedures  are  a  vital  par 
of  the  Quality  Assurance  System.  Written 
procedures  describing  such  items  as  workfloi 
routes;  sources  utilized  for  inspections; 
listings  of  certifications,  qualifications 
and  laboratory  approvals  by  other  agencies; 
detailed  procedures  for  conducting  tests 
analyses,   inspections,  etc.;  and  detailed 
procedures  for  sampling  or  selecting  speci- 
mens for  testing,  are  all  a  necessary  part 
of  operations. 

Personnel 

Personnel  is  a  major  key  to  establish- 
ment of  a  Quality  Assurance  System  in  a 
laboratory.     It  would  really  be  great  if  we 
could  all  have  a  cadre  of  perfect  personnel 
but  I  don't  think  that  anyone  has  achieved 
this  perfect  work  force.     Personnel  manage- 
ment systems  therefore  needed,  are  Job 
Responsibility  Mandates   (job  descriptions), 
Job  Training  Programs,  Personnel  Classifi- 
cations and  Personnel  Biographies. 
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Equipment  and  Calibration 

In  order  to  establish  credibility 
in  the  field  of  testing,  all  equipment 
used  must  be  accurate,  reliable  and  de- 
pendable.    This  constitutes  the  physical 
facilities  of  the  laboratory.     A  com- 
plete list  or  inventory  of  all  equip- 
ment requiring  calibration  or  standard- 
ization must  be  maintained;  sources  for 
calibration  and  required  frequency  must 
be  kept  updated  constantly  and  those  en- 
vironmental conditions  required  must  be 
met . 

Now  this  is  where,  as  they  say, 
the  rubber  meets  the  road.     The  key- 
stone to  all  systems,  Quality  Assurance, 
Quality  Control,  Accreditation,  etc., 
etc.,   is  DOCUMENTATION.     Unless  it's 
recorded,  it  has  no  value.  Records 
must  be  maintained  and  corrective  ac- 
tions verified.     Suitable  records  shall 
be  maintained  for  calibratable  labora- 
tory standards  and  test  equipment, 
whether  used  internally  or  externally. 
Each  inventory  listing  should  contain 
the  following: 

1.  The  name  of  the  manufacturer. 

2.  The  equipment  model  and  serial 
number. 

3.  The  properties  subject  to 
standardization . 

4.  The  range  of  operation  and  the 
range  of  calibration. 

5.  A  reference  to  a  recognized 
calibration  procedure. 

6.  The  frequency  of  calibration. 

7.  Allowable  tolerances  or  maxi- 
mum sensitivity. 

8.  The  source  of  verification. 


suitability  -  shall  be  established  by 
each  laboratory.     Subsequent  to  the 
cross-referencing  of  test  equipment  and 
calibration  standards,  the  necessary 
sources  and  required  frequency  for  ver- 
ification shall  be  established.     The  suc- 
cess of  the  calibration  system  is  pri- 
marily dependent  upon  the  quality  con- 
trol supervisor  who  is  responsible  for 
scheduling  calibrations  or  standardiza- 
tions.    An  individual  selected  for  this 
position  should  be  personally  responsi- 
ble and  able  to  delegate  authority,  but 
not  responsibility,  to  subordinates.  De- 
termination of  realiztic  calibration  in- 
tervals is  dependent  upon  the  basis  of 
stability  of  the  instrument  or  standard, 
its  purpose  and  degree  of  usage. 

Certain  calibrations  and  test  meth- 
ods require  controlled  environmental 
conditions  to  insure  the  accuracy  of  re- 
sults.    Dependent  upon  the  physical  prop- 
erty and  degree  of  sensitivity  of  the 
calibration  or  test,  certain  environ- 
mental factors  such  as  temperature,  hu- 
midity, cleanliness,  vibration,  voltage, 
radio  frequency  interference,  pressure 
and  atmosphere  must  be  controlled.  The 
extent  of  the  control  required  however 
should  be  specified. 

Written  procedures  and  documented 
evidence  of  calibrated  standards  and  test 
equipment  provide  the  basis  for  a  sound 
Quality  Control  Program.     Procedural  de- 
tail may  be  individually  derived  by  each 
laboratory  or  may  be  altered  to  conform 
to  established  recognized  standards. 
Primarily  the  scope  of  the  operation 
must  satisfy  the  referee  or  inspector, 
as  well  as  conform  to  the  Client's  re- 
quirements.    Sufficient  detail  should 
be  incorporated  into  an  in-house  proce- 
dure so  as  to  make  the  document  instruc- 
tional to  internal  personnel  as  well  as 
meaningful  to  external  referees.  The 
following  information  is  pertinent  to 
such  a  document: 


9.     A  chronological  history  of  re- 
pairs, modifications  or  sub- 
stitutions . 

10.     Traceability  of  reference 

standards  to  the  National  Bur- 
eau of  Standards  or  accepted 
values  of  natural  physical 
constants . 

The  proof  of  adequacy  of  standards, 
and  test  equipment  in  particular  -  cap- 
ability for  accuracy  -  stability  and 


Type  of  calibration  or  stand- 
dardization  being  performed. 

Specifics  such  as  calibration 
points  or  intervals,  accuracy 
required,  governing  specifica- 
tions, etc. 

Necessary  standards  and  equip- 
ment . 

Record  of  test  data. 
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5.  Equipment  condition. 

6.  Final  disposition. 

Measuring  and  test  equipment  shall 
be  calibrated  by  the  laboratory  or  an 
external  commercial  facility,  either  of 
which  must  utilize  reference  standards 
whose  calibration  has  been  certified  as 
being  traceable  to  the  National  Bureau 
of  Standards  or  has  been  derived  from 
accepted  values  of  natural  physical  con- 
stants.    Certificates  or  reports  shall 
attest  to  the  fact  that  the  standards 
used  in  obtaining  the  results  have  been 
compared  at  planned  intervals  with  a 
standard  traceable  to  the  national 
standard  or  with  independently  repro- 
duced standards. 

To  assure  continuity  of  satisfac- 
tory operations,  provisions  must  be 
made  for  the  prevention  of  equipment 
and  personnel  inaccuracies.     A  docu- 
mented system  of  existing  practices  in- 
cluding periodic  inspections  or  opera- 
tional checks  to  insure  stability  be- 
tween calibrations  shall  be  established 
to  assure  uniformity  of  performance  and 
to  serve  as  a  basis  for  alteration  of 
standardization  intervals. 

Records  provide  objective  evidence 
that  calibration  schedules  are  complied 
with  and  that  the  accuracy  of  the  equip- 
ment or  standards  is  being  maintained. 
Proof  of  performance  should  include 
reference  standard  certificates,  data 
sheets  containing  actual  values  of  both 
present  and  prior  calibrations,  a  sum- 
mary of  routine  maintenance  and  repairs 
made  and  the  final  calibration  form 
which  suitably  cross-references  the 
equipment  to  the  reference  or  transfer 
standard. 

Reference  samples 

After  all  the  personnel,  equipment 
and  calibration  systems  are  in  order  and 
are  being  documented,   then  the  job  of 
Quality  Assurance  is  still  not  completed. 
It  is  a  well  known  fact  in  statistical 
analyses  that  variables  do  occur,  that 
makes  results  fall  outside  of  accepted 
limits.     We  must  therefore  establish  con- 
fidence limits.     The  true  indication  of 
a  good  Quality  Control  Program  is  in  the 
results  obtained;  accurate,  reliable,  re- 
peatable  results.     This  is  in  effect 
"the  proof  is  in  the  pudding",   and  is 
determined  by  objective  and  competent 
personnel  using  standard  methods  and  ac- 


curate equipment,  instruments,  and  mate- 
rials to  ascertain  the  repeatability  and 
accuracy  of  the  laboratories'   test  re- 
sults.    This  proof  is  obtained  by  the  use 
of  reference  samples  an/or  inspection 
services.     There  is  a  need  in  this  area 
for  the  establishment  of  "round-robin" 
testing  and  comparative  results.  There 
are  many  organizations  capable  of  origin- 
ating and  performing  this  service,  but  at 
this  time  more  is  needed. 

Impact  of  the  Manual  on  the  Labora- 
tory Community 

To  the  uninitiated  in  the  ways  of 
Quality  Control  the  first  impact  upon 
starting  is  ghastly  horror.     It  is  "hor- 
rendous" job  to  develop  a  Quality  Assur- 
ance Program  starting  from  point  zero. 
It  is  a  lot  of  work  and  takes  consider- 
able time.     It  is  costly,  and  there  is  no 
use  kidding  you  about  it.     Those  who  have 
done  it  know  only  too  well  what  a  project 
it  is.     Furthermore,  it  is  a  continuing 
job,  an  ongoing  project  that  needs  con- 
stant attention.     Once  established,  how- 
ever, the  Quality  Assurance  Program  can 
be  and  is  very  beneficial.     Its  use  can 
be  beneficial  in  sales  of  services,  it 
is  irreplaceable  in  accreditation  pro- 
grams and  benefits  are  boundless  in  loss 
prevention  programs. 

The  Quality  Control  Manual  is  an 
instruction  kit  that  you  have  written  for 
yourself  to  guide  your  laboratory  opera- 
tion in  the  production  of  quality  work. 
I  have  seen  some  of  the  manuals  produced 
by  laboratories  and  they  are  excellent. 
When  completed,  a  Manual  may  well  be  con- 
sidered to  be  a  proprietary  document,  due 
essentially  to  the  cost  that  it  has  taken 
to  produce  it.     Many  are  also  considered 
to  be  proprietary  due  to  the  content  that 
a  laboratory  has  included  for  its  own  use 
but  not  intended  for  public  dissemina- 
tion.    I  do  not  feel  that  I  am  at  liberty 
to  even  suggest  the  names  of  laboratories 
that  I  know,   that  have  completed  their 
manuals,  but  if  you  inquire  you  might 
find  some  laboratory  that  is  willing  to 
share  its  manual  with  you. 

Use  of  the  Manual  of  Practice 

I  personally  feel  extremely  pleased 
that  the  Manual  of  Practice,  Quality  Con- 
trol System  has  been  accepted  by  many 
laboratories  and  also  by  many  agencies 
now  writing  accreditation  criteria.  It 
has  become  the  backbone  of  many  systems 
now  in  operation.     I  come  from  a  genera- 


102 


tion  of  laboratory  personnel  who  needed 
no  quality  control  because  they  were  the 
experts  -  their  word  was  final  -  if  they 
said  so,  that  was  it.     Well,  times  have 
changed  -  thank  God.     And  now  when  we  say 
it,  we  have  the  documentation  to  prove 
it,  we  have  the  system  to  back  us  up. 

The  use  of  the  Quality  Assurance 
System  in  day-to-day  operations  of  the 
laboratory  is  of  the  utmost  importance. 
Operational  checks  of  equipment  before 
the  day's  work  begins,  the  check  of  in- 
struments and  the  constant  use  of  stand- 
ards have  done  more  to  upgrade  testing 
and  inspection  services  than  we  can 
realize.     And  this  is  only  the  beginning. 

The  broader  impact  of  the  Quality 
Assurance  System  is  in  the  accreditation 
process.     The  certification  or  accredi- 
tation of  laboratories  or  inspection 
agencies  is  a  failure  without  recourse 
to  the  Quality  Assurance  System.  Stud- 
ies have  shown  that  a  good  Quality  Assur- 
ance System,  with  proper  documentation, 
,    provides  the  basic  need  for  the  accredi- 
tation audit  needed. 
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LABORATORY  QUALITY  PROGRAM  REQUIREMENTS 
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The  need  for  quality  control  of  laboratory  results  was  identified 
early  by  The  National  Institute  for  Occupational  Safety  and  Health  when 
a  great  disparity  of  analytical  results  was  noted  among  14  industrial 
hygiene  laboratories.    The  result  was  a  four  pronged  attempt  to  improve 
performance:    1  -  An  educational  program  was  developed;    2  -  An  inter- 
laboratory  proficiency  program,  PAT,  was  started;    3  -  A  laboratory 
quality  control  manual  and  a  standard  for  laboratory  control  were 
developed;    4  -  A  laboratory  accreditation  program  was  developed  by  AIHA 
under  contract. 

The  Environmental  Protection  Agency,  the  Occupational  Safety  and 
Health  Administration,  the  Food  and  Drug  Administration,  the  American 
Society  for  Testing  Materials  and  others  have  described  similar  programs 
in  various  standards  and  regulations.    Finally  the  American  National 
Standards  Institute  (ANSI)  Z-l  Committee  on  Quality  Assurance  and  its 
Systems  and  Procedures  Sub-Committee  have  agreed  that  there  is  a  need 
for  a  generic  Quality  Control  Standard  for  laboratories  which  will  differ 
from  a  manufacturing  quality  control  standard.    A  laboratory  quality 
control  standard  is  being  drafted  by  a  writing  committee  under  the  aegis 
of  the  Biomedical  Division  of  the  American  Society  for  Quality  Control. 

Keywords:    Accuracy;  control  chart;  control  limits;  corrective 
action;  data  validation;  precision;  proficiency  analytical  testing; 
quality  control;  statistical  quality  control. 


1.  Introduction 

Under  the  Occupational  Safety  and  Health 
Act  of  1970,  the  National  Institute  for  Oc- 
cupational Safety  and  Health  was  given  the 
responsibility  for  organizing  the  analyzing 
and  the  reporting  of  samples  collected  by 
OSHA  Compliance  officers  in  the  detection  of 
contaminants  in  industrial  atmospheres. 

Under  the  provisions  of  Section  7(c)(1) 
of  the  above  Act,  contracts  were  let  by  OSHA 
with  12  state  industrial  hygiene  laboratories 
to  assist  in  handling  the  work  load. 

Early  in  the  program,  to  determine  the 
level  of  agreement  among  14  laboratories,  12 


state  and  2  NIOSH,  known  samples  of  contami- 
nants were  sent  to  the  participating  organi-l 
zations.    Not  surprisingly,  14  different 
results  were  reported. 

Accordingly,  a  number  of  steps  were  J 
taken  in  an  effort  to  improve  laboratory 
performance  by  obtaining  better  agreement  | 
among  laboratory  results.    To  put  it  another 
way,  a  concerted  effort  was  made  to  improve 
the  quality  of  laboratory  product,  the  "pro-; 
duct"  being  reports  of  analytical  results.  t 
It  was  sought,  therefore,  to  improve  pre- 
cision and  accuracy  of  method  determination 
reports,  while  not  providing  methods  evalu- 
ation information. 
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2.    Quality  Improvement  Measures 


2.3    Laboratory  Accreditation 


Over  a  period  of  time  NIOSH  has  taken 
the  following  steps  to  improve  laboratory 
performance  quality: 

2.1  Proficiency  Analytical  Testing 

Program 

NIOSH  started  the  Proficiency  Analytical 
Testing  Program  in  May,  1976  known  as  PAT.[1] 
The  Chemical  Reference  Laboratory,  in  Cin- 
cinnati, by  contract  prepares  known  samples 
of  contaminants  --  asbestos,  lead,  silica 
dust,  cadmium  and  zinc,  and  one  of  eight 
solvents  on  charcoal  collection  tubes.  These 
are  mailed  at  two  month  intervals  to  partici- 
pating laboratories.    Results  are  returned  to 
the  CRL.    There  they  are  tabulated  and  for- 
warded to  the  participant  laboratories. 

Each  round  has  four  sets  of  samples 
(subgroups)  plus  one  blank.    We  are  currently 
preparing  control  charts  similar  to  (Figure  1) 
for  each  laboratory  and  each  contaminant. 
Plotted  control  limits  help  give  an  indica- 
tion of  drift  tendencies,  and  a  picture  of 
whether  or  not  labs  were  running  their  deter- 
minations out  of  control.    Current  data  are 
used  to  determine  statistically  valid  control 
limits. 

In  addition,  selected  labs  under  contract 
have  been  given  "hands-on"  assistance  in  set- 
ting up  and  maintaining  a  quality  control 
program.    This  assistance  was  given  by  a  con- 
sulting firm  under  contract  to  NIOSH. 

2.2  NIOSH  Laboratory  Quality 

Control  Manual 

In  order  to  improve  and  make  consistent 
the  quality  of  results  obtained  by  the  lab- 
oratory the  NIOSH  Industrial  Hygiene  Service 
Laboratory  Quality  Control  Manual  (Technical 
Report  #78)[2]  was  published  and  widely  dis- 
tributed.   This  draft  manual,  which  identifies 
and  describes  eleven  elements  of  a  laboratory 
quality  control  program  serves  two  purposes. 
It  assists  small  laboratories  in  setting  up 
a  quality  program,  including  preparation  of 
a  quality  manual,  and  it  provides  text  mater- 
ial for  courses  given  by  the  Training  and 
Manpower  Development  Division  of  NIOSH. 

We  have  found  that  even  in  a  very  small 
organization,  four  or  five  people  can  suc- 
cessfully conduct  a  viable  quality  program. 


NIOSH  has  also  contracted  with  the 
American  Industrial  Hygiene  Association  to 
support  preparation  and  validation  of  an 
accreditation  program  for  industrial  hygiene 
analytical  laboratories.    This  accreditation 
includes  among  other  things,  the  need  to 
establish  and  maintain  a  quality  control  sy- 
stem within  the  laboratory. 

2.4  Quality  Training 

The  Training  and  Manpower  Development 
Division  of  NIOSH  periodically  offers  a 
course  to  assist  interested  laboratories  in 
obtaining  accreditation.    A  substantial  por- 
tion of  this  three-day  presentation  is  de- 
voted to  quality  control  matters,  as  can  be 
seen  from  the  following  course  subjects 
covered: 

History  of  Laboratory  Accreditation 
The  Current  Program 
Criteria  for  Accreditation 
Accreditation  Applications 
Proficiency  Analytical  Testing 
Intralaboratory  Quality  Control 
Frequency  Distribution  and  Measures  of 

Control  Tendency  and  Variability 
Measuring  Variability 
The  Normal  Curve  and  Analysis  of 

Control 
Control  Charts 
X  -  R  Control  Charts 
Problem  Session  -  Control  Charts 
Problem  Session  Review  -  Control  Charts 
Specific  Applications  of  Quality  Control 

Charts 

Special  Situation  Control  Charts 
Data  Tests  and  Other  Statistical  Tech- 
niques 

2.5  Laboratory  Quality  Control 

Specification 

NIOSH  has  developed  a  Laboratory  Quality 
Control  Specification  entitled,  "Industrial 
Hygiene  Laboratory  Quality  Program  Require- 
ments". [3]    It  identifies  22  elements  needed 
in  a  laboratory  quality  control  program  as 
follows : 

Statement  of  Objectives 
Policy  Statements 
Organization 
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Quality  Planning 

Standard  Operating  Procedures 

Recordkeeping 

Chain-of-Custody  Procedures 
Corrective  Action 
Qua! ity  Training 
Document  Control 
Cal ibration 

Preventive  Maintenance 

Reagent  and  Reference  Standards 

Procurement  and  Control 

Sample  Identification  and  Control 

Laboratory  Analysis  and  Control 

Interlaboratory  and  Intralaboratory 

Testing  Programs 
Handling,  Storage  and  Delivery 
Statistical  Quality  Control 
Data  Validation 
System  Audits 

It  can  readily  be  seen  that  though  there 
are  some  similarities  to  a  manufacturing 
quality  control  system,  many  of  these  elements 
are  peculiar  to  the  laboratory  environment. 

2.6    Other  Laboratory  Quality 
Control  Activity 

While  NIOSH  was  engaging  in  these  attempts 
to  improve  laboratory  quality,  other  organi- 
zations were  concerned  about  similar  problems 
and  undertook  various  steps  to  alleviate  them. 
The  U.S.  Environmental  Protection  Agency 
Quality  Assurance  and  Environmental  Monitoring 
Laboratory  is  publishing  a  monumental  three- 
volume  "Quality  Assurance  Handbook  for  Air 
Pollution  Measurement  Systems"^]     Volume  I  - 
"Principles"  is  in  print  and  it  identifies  and 
discusses  in  detail  23  elements  of  Quality 
Assurance  as  follows:    Volumes  II  and  III  are 
in  preparation. 

ELEMENTS  OF  QUALITY  ASSURANCE 

Document  Control  and  Revisions 

Quality  Assurance  Policy  and  Objectives 

Organi  zation 

Quality  Planning 

Training 

Pre-Test  Preparation 
Preventive  Maintenance 
Sample  Collection 
Sample  Analysis 
Data  Reporting 
Procurement  Quality  Control 
Cal ibration 
Corrective  Action 
Quality  Costs 

Interlaboratory  and  Intralaboratory 
Testing 


Audit  Procedures 

Data  Validation 

Statistical  Analysis  of  Data 

Configuration  Control 

Reliability 

Quality  Reports  to  Management 
Quality  Assurance  Manual 
Quality  Assurance  Plans  for  Projects 
and  Programs 

The  Mine  Safety  and  Health  Administra 
tion  is  carrying  out  an  active  inter- 
laboratory test  program  to  standardize  the 
accuracy  of  reference  standard  gas 
concentrations  used  in  calibrating  laborat 
gas  chromatographs . 

In  September  1973,  The  Occupational 
Safety  and  Health  Administration  published  : 
Title  29  Code  of  Federal  Regulations,  Partj 
1907  Accreditation  of  Testing  Laboratorie! , 
This  regulation,  now  temporarily  revoked,  j.s 
a  double  requirement  for  quality  control. 
First,  testing  laboratories,  testing  and 
approving  a  wide  range  of  safety  and  healt 
related  equipment  from  ladders  to  fire  ex- 
tinguishers, must  demonstrate  existence  of 
a  satisfactory  quality  control  system  in 
order  to  gain  accreditation.    In  turn,  the 
accredited  testing  laboratory  must  demand 
that  its  clients,  who  are  manufacturers  of 
products  over  which  OSHA  excercises  sur- 
veillance, have  an  effective  quality  contr 
system,  ensuring  this  by  periodic  quality 
surveys  and  audit  of  off-the-shelf  purchas 
of  their  products. 


While  these  and  other  government  agenc 
were  developing  and  attempting  to  use  thes 
relatively  new  laboratory  quality  control 
techniques,  two  other  groups  were  beginnin 
to  exhibit  interest  in  this  area.    The  fir 
was  what  was  formerly  the  Biomedical  Contr 
Technical  Committee  of  ASQC.  (Now  the  Bio-t 
medical  Division).    Early  in  1973,  NIOSH 
established  liaison  with  this  organization 
which  now  has  a  laboratory  quality  control 
writing  committee  of  which  the  author  is  a 
member.    NIOSH  has  solicited  and  received 
from  this  committee  outstanding  interest 
and  assistance  in  the  preparation  of  the 
Laboratory  Quality  Control  Specification 
noted  above. 

A  primary  interest  of  this  ASQC  commit 
is  in  the  preparation  of  a  laboratory  qua! 
assurance  standard  and  work  is  proceeding 
this  end. 
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Secondly,  the  organization  meeting  of 
the  American  National  Standards  Institute 
Committee  Z-l  on  Quality  Assurance  was  held 
on  February  5,  1975.    Within  the  Z-l  Commit- 
tee there  is  the  systems  and  procedures 
sub-committee.    One  of  the  functions  of  this 
committee  is  to  identify  the  need  for  one  or 
more  generic  ANSI  standards  for  the  manage- 
ment of  the  quality  system. 

The  American  Society  for  Testing  Material 
has  published  in  draft  a  "Recommended  Prac- 
tice for  Laboratories  Engaged  in  Sampling 
and  Analysis  of  Atmospheres".    Its  require- 
ments for  Laboratory  Quality  Control  are 
similar  to  those  of  the  NIOSH  quality 
specification. 

I  am  sure  that  all  present  are  familiar 
with  the  NAVLAP  provisions  for  laboratory 
qual ity  control . 

The  FDA  regulations  21  CFR  1000  "Diagnos- 
tic Radiology  Facilities  Quality  Assurance 
Programs"  and  21  CFR  82  "Medical  Devices" 
provide  typical  examples  of  regulatory 
requirements  for  laboratory  quality  control. 

Since  laboratories  of  all  kinds  fall 
generally  under  the  service  industry  cate- 
gory, NIOSH  feels  that  its  "Specification  for 
an  Industrial  Hygiene  Laboratory  Quality 
Control  System"  can  with  some  judicious 
editing  and  pruning,  be  made  applicable  to 
the  operations  of  any  type  of  laboratory 
and,  by  extension,  to  any  type  of  service 
industry. 

The  foregoing  has  been  an  explanation  of 
the  efforts  of  various  governmental  indus- 
trial and  standards  publishing  individuals 
and  groups  to  develop  and  agree  upon  an 
acceptable  quality  control  system  standard 
for  laboratories  of  every  kind.    Once  one 
has  accepted  the  basic  premise  that  all 
laboratories  receive  samples  of  some  kind 
from  some  source;    must  identify  and  keep 
track  of  them;    analyze  or  test  them;  re- 
port on  results;    keep  adequate  records; 
maintain  the  accuracy  of  test  and  analytical 
equipment  by  systematic  calibration  and 
carry  out  all  or  most  of  the  quality  control 
elements  listed  as  necessary  above,  then  it 
seems  "as  follows  night,  the  day"  that  a 
laboratory  quality  control  system  standard 
is  not  only  necessary  but  feasible  and 
finally,  inevitable. 


[3] 


[4] 
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Figure  1  -  Sample  Control  Chart  For  Each  Laboratory 
For  Each  Contaminant 
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QUALITY  CONTROL  PROCEDURES  IN  A  LABORATORY 
TESTING  CEMENT  FOR  COMPRESSIVE  STRENGTH 

RICHARD  D.  GAYNOR  AND  RICHARD  C.  MEININGER 

NATIONAL  READY  MIXED  CONCRETE  ASSOCIATION 
900  SPRING  STREET,   SILVER  SPRING,  MARYLAND  20910 

This  paper  describes  an  in-house  quality  control  system 
used  by  the  Joint  Research  Laboratory  (JRL)  of  the  National 
Ready  Mixed  Concrete  Association  and  the  National  Sand  and 
Gravel  Association  (NRMCA-NSGA) .     The  system  is  used  to  control 
the  quality  of  the  results  of  compressive  strength  tests  of 
Portland  or  blended  cements  by  ASTM  Method  C109  but  can  also 
be  used  for  other  physical  and  chemical  tests  of  cement  and 
concrete. 

Key  words:     Cement  and  Concrete  Reference  Laboratory;  cement 
and  concrete  testing;  compressive  strength;  control  charts; 
in-house  quality  control;  quality  control  system. 

Introduction 


The  NRMCA-NSGA  Joint  Research  Laboratory 
routinely  participates  in  the  Cement  Refer- 
ence Sample  Program  of  the  NBS  Cement  and 
Concrete  Reference  Laboratory  (CCRL)  in  which 
pairs  of  cement  samples  are  distributed  twice 
a  year.     In  addition  the  JRL  has  obtained  a 
large  number  of  sealed  samples  of  a  local 
cement  which  are  tested  as  a  "standard"  or 
"reference"  cement  whenever  an  "unknown 
cement"  is  tested  and  when  the  CCRL  samples 
are  tested.     The  paper  describes  the  system 
used  to  control  the  quality  of  tests  for 
cement  compressive  strength  by  ASTM  C109. 
The  system  can  also  be  used  for  other  physi- 
cal and  chemical  tests  of  cement. 

Control  charts  are  maintained  of  the 
deviation  of  JRL  results  from  the  average 
of  other  laboratories  participating  in  the 


Cement  Reference  Sample  Program  and  of  the 
deviation  of  the  reference  cement  from  es- 
tablished averages.     The  data  generated 
demonstrate  that  in  the  JRL  the  most  trou- 
blesome variations  are  those  which  occur 
over  long  periods  of  time.     Standard  devi- 
ations of  tests  made  within  a  period  of 
several  days  are  approximately  half  as  large 
as  the  standard  deviations  of  tests  made 
over  a  period  of  several  years.     Tests  of 
standard  cement  made  at  the  same  time  as 
the  CCRL  reference  samples  show  excellent 
correlation.     That  is,  if  the  standard 
cement  results  are  high,  results  on  the 
CCRL  reference  samples  will  also  be  high 
relative  to  the  average  of  all  laboratories 
in  the  program. 
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CCRL  Cement  Reference  Sample  Program 

Since  1965  the  CCRL  has  distributed 
pairs  of  cement  samples  every  six  months  to 
a  group  of  120  to  180  participating  labora- 
tories. Each  laboratory  is  asked  to  perform 
physical  and  chemical  tests  in  accordance 
with  designated  ASTM  Standard  methods.  The 
cement  reference  sample  program  is  described 
in  papers  by  several  authors . [1, 2 , 3 ,4] 1  Re- 
sults are  reported  to  the  CCRL  and  in  a 
short  time  each  laboratory  receives  a  consol- 
idated report  giving  the  average  values ,  mul- 
ti-lab precision,  single  lab  precision  and 
other  statistical  data  along  with  scatter 
diagrams  for  each  test.     Further,  each  lab- 
oratory receives  a  numerical  rating  from  5 
to  1  depending  upon  the  deviation  of  its  re- 
sult from  the  average  of  all  other  labora- 
tories.*   The  rating  scale  is  in  multiples 
of  the  multi-lab  standard  deviation  in  ac- 
cordance with  the  schedule  in  Table  1. 

Use  of  the  Cement  Reference  Sample  Data 

Although  the  cement  reference  sample 
data  have  been  useful  in  the  development  of 
precision  statements  in  several  ASTM  methods 
and  in  evaluating  methods  and  procedural  var- 
iations there  has  been  little  written  des- 
cribing how  a  laboratory  can  use  the  data  to 
improve  its  own  performance  or  control  test 
procedures,  aside  from  the  examination  of 
the  scatter  diagrams  and  the  tabulation  of 
the  numerical  ratings  from  5  to  1  described 
earlier. 

Table  1  summarizes  the  ratings  of  the 
JRL  (Laboratory  112) for  3  and  7  day  compres- 
sive strength  for  C  109  mortars. 

In  the  CCRL  program  two  samples  of  ce- 
ment are  distributed  to  participants  twice  a 
year.     For  compressive  strength  each  labora- 
tory is  asked  to  prepare  a  single  batch  of 
C  109  mortar  with  each  cement  and  to  mold 
six  2  inch  cubes  for  test  in  compression. 

For  all  samples  except  numbers  9-12, 
three  cubes  are  broken  at  3  days  and  three 
at  7  days.     Samples  9  to  12  were  Type  III 
cement  and  the  test  ages  were  1  and  3  days. 
Since  the  JRL  does  not  perform  all  of  the 
physical  and  chemical  tests  the  samples  are 
large  enough  for  the  main  operator  to  mix 
two  batches  from  each  sample.     Generally  one 
additional  operator  makes  an  additional 
batch  as  a  part  of  a  continuing  operator 
training  program.  Only  the  results  of  the 
first  batch  mixed  are  reported  to  the  CCRL. 


lFigures  in  brackets  indicate  the  litera- 
ture references  at  the  end  of  the  paper. 
*Using  CCRL  multi-lab  standard  deviation. 


In  1968  the  JRL  secured  a  supply  of 
carefully  blended  cement  and  sealed  a  large 
number  of  samples  in  metal  cans  to  provide  e 
additional  "standard  cement,"  Lot  4538.  Th£ 
cement  is  tested  with  each  CCRL  sample  and 
whenever  an  important  unknown  sample  is  re 
ceived  for  testing. 

Table  2  is  the  detailed  compressive 
strength  data  for  tests  of  samples  51,  52, 
and  the  standard  cement.  Generally,  Batch  A 
is  mixed  on  one  day  and  Batch  B  on  the  folic 
ing  day.  Method  C  109  defines  a  "test"  as  tl. 
average  of  tests  of  3  cubes  and  although  va 
ues  for  individual  cubes  are  reported  to  the 
CCRL  only  the  average  values  are  used  in  the 
CCRL  reports  and  tabulations.  Table  A  is  a 
summary  of  the  deviation  of  the  JRL  test 
values  from  the  average  for  all  participati 
laboratories.  Table  B  is  a  similar  tabula- 
tion for  the  standard  cement,  Lot  4538. 
Tables  A  and  B  are  appended  to  this  paper 

Table  3  is  a  summary  of  various  aver 
ages  and  standard  deviations  calculated  fro 
the  strength  deviations  in  Tables  A  and  B 
Figure  1  shows  a  good  correlation  between  t 
deviations  from  the  CCRL  sample  averages  an 
the  deviations  of  tests  of  Lot  4538  standar 
cement  mixed  at  the  same  time.  The  Lot  453 
deviations  are  calculated  from  the  average 
used  for  that  lot.  Plotted  values  are  for 
tests  of  individual  batches.  The  line  show 
is  drawn  through  an  overall  average  CCRL 
deviation  of  -10  psi  and  an  average  Lot  453 
deviation  of  -75  psi  and  is  parallel  to  the 
line  of  equality.  No  regression  analysis 
has  been  made. 


; 
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The  overall  variation  is  calculated  as 
the  standard  deviation  of  the  individual 
batch  deviations     in  the  respective  tables 
taking  into  account  the  sign  of  each  devi- 
ation.    The  average  deviation,  -26  and 
+6  psi  for  the  CCRL  samples  are  negligible 
and  near  zero. 

For  Lot  4538  the  deviation  averaged  -' 
and  -74  psi.  These  are  barely  of  statisti- 
cal significance.  The  corresponding  aver- 
ages were  zero  after  sample  40.  The  overall 
standard  deviations  range  from  182  to  229  ps 
The  differences  between  standard  deviation! 
for  CCRL  and  Lot  4538  samples  are  not  sig- 
nificant. Further,  although  the  7-day 
values  appear  larger  they  are  not  signifi- 
cantly higher  than  the  3-day  values.  A 
representative  value  for  the  overall  long- 
term  standard  deviation  would  be  about  212 
psi. 
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The  short  term  batch-to-batch  standard 
deviation  from  batches  mixed  on  consecutive 
days  can  be  estimated  from  the  range  of 
strength  test  results  obtained  on  Batches 
"A"  and  "B"   (or  the  range  of  the  deviations). 
These  ranges  and  their  averages  are  given  in 
Tables  A  and  B.     Table  3  gives  the  standard 
deviations  and  averages  values  for  the  CCRL 
and  Lot  4538  tests.     Again  the  values  for 
3-  and  7-day  tests  are  not  significantly 
different,  and  a  standard  deviation  of  98 
psi  is  representative  of  the  short  term  vari- 
ation at  either  age. 

The  short  term  or  between  batch  standard 
deviation  ranges  from  2.5  to  3.8  percent  of 
the  average  for  Lot  4538  and  can  be  compared  to 
the  single  laboratory  standard  deviation  con- 
tained in  the  precision  statement  for  Method 
C  109  of  3.8  percent.     Note  that  the  C  109 
precision  statement  was  derived  from  the  ce- 
ment reference  sample  program  data. 

The  overall  standard  deviations  are 
roughly  7  and  5.5  percent  of  the  average  and 
are  similar  in  size  to  the  C  109  multi-labor- 
atory coefficient  of  variation  of  7.3  percent. 

Construction  of  Control  Charts 

As  noted  in  the  previous  section  the 
overall  or  long  term  standard  deviation  is 
over  twice  the  day-to-day  standard  devia- 
tion and  even  if  the  day-to-day  component 
could  be  eliminated  the  remaining  long  term 
standard  deviation  would  exceed  170  psi.  In 
marked  contrast  with  typical  cement  company 
laboratories  which  test  cement  every  day  the 
JRL  often  tests  only  a  few  cements  in  the  6 
month  interval  between  reference  samples. 
Presumably  this  relatively  small  amount  of 
practice  makes  it  difficult  to  avoid  the 
long  term  procedural  variations.. 

Until  additional  analysis  was  under- 
taken for  this  paper  the  JRL  maintained  con- 
trol charts  on  the  average  deviation  of  two 
batches  with  3s  and  2s  control  limits  based 
on  the  overall  standard  deviation.  Limits 
were  symmetrical  about  the  average  deviation. 

At  the  present  time  a  decision  has  been 
made  to  maintain: 

a.  Separate  3-and  7-day  control  charts  for 
the  reference  samples.   (Deviation  from 
CCRL  average. ) 

b.  A  control  chart  for  3-and  7-day  tests  of 
Lot  4538.     (Deviation  from  Lot  4538  av- 
erage. ) 


c.  A  difference  2  sigma  limit  for  acceptable 
agreement  between  duplicate  batches  appli- 
cable to  all  usual  cements. 

d.  A  difference  2  sigma  limit  for  acceptable 
agreement  between  the  three  cubes  that  are 
averaged  to  constitute  a  test. 

Control  Charts  for  Reference  Samples 

Figures  2  and  3  are  the  control  charts 
for  tests  of  cement  reference  samples. 
Strength  is  expressed  as  a  deviation  from  the 
CCRL  average  for  all  laboratories.  Each 
point  represents  the  average  for  two  batches. 
The  chart  line  connects  the  average  deviation 
obtained  on  each  pair  of  samples  distributed 
in  the  program.     Three  sigma  control  limits 
are  given  for  the  average  deviation  for  the 
two  batches  mixed  for  each  sample  and  the 
average  of  the  pair  of  samples  distributed  at 
a  given   time    (4  batches).    The  standard  de- 
viation used  is  that  for  batches  mixed  over  a 
short  period.     The  limits  are  computed  re- 
spectively as : 

0  ±  3s/ (2) %  for  2  batches 

0  ±  3s/ (4)^   for  4  batches 

Note  that  laboratories  which  do  not  have 
enough  cement  to  test  duplicate  batches  can 
likely  estimate  the  short  term  standard  de- 
viation from  the  range  of  deviations  of  the 
pair  of  samples  mixed  in  the  program.  Suit- 
able values  could  also  be  derived  from  dupli- 
cate batches  of  a  reference  cement  or  even 
from  the  C  109  precision  statement. 

Figure  4  is  the  control  chart  for  3  and 
7  day  tests  of  the  standard  cement  Lot  4538. 
Again  construction  is  similar  and  the  short 
term  standard  deviation,  averaged  for  3-and 
7-day  tests,  is  used. 

Examination  of  Control  Charts , 
Figures  2,  3  and  4 

1.  Low  results  were  obtained  on  reference 
samples  5  and  6  and  this  is  the  primary 
reason  why  the  standard  Lot  4538  cement 
was  used  with  sample  7  and  all  later 
samples . 

2.  Samples  11  and  12  produced  markedly  di- 
vergent results ,  but  the  reference  cement 
tests  were  normal.     It  is  clear  now  that 
the  sample  numbers  were  interchanged  since 
the  average  for  the  sample  pair  is  reason- 
ably good. 
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3.  No  ready  explanation  is  available  for  the 
high  results  on  samples  17  and  18  and  only 
moderately  high  results  on  Lot  4538. 

4.  High  results  were  produced  in  all  tests  of 
samples  21  and  22. 

5.  Unfortunately  no  reference  cement  was 
tested  in  the  period  from  sample  29  to  34. 
During  this  period  the  laboratory  was  in 
temporary  quarters  before  moving  into  new 
facilities  in  1974. 

6.  Samples  31  and  32  produced  low  7-day 
strength  and  high  28-day  strengths.  No 
good  explanation  was  discovered. 

7.  Starting  with  sample  39-40  a  formal  con- 
trol chart  procedure  and  on-going  anal- 
yes  were  implemented.  Agreement  between 
batches  and  samples  greatly  improved. 
Strength  levels  in  all  tests  decreased  pro- 
gressively through  samples  51-52.  Proce- 
dures were  thoroughly  reviewed.  Training 
started  on  a  new  backup  operator.  Tests 
demonstrated  that  cube  density  had  decreased 
and  that  increased  tamping  pressure  would 
increase  both  density  and  strength.  When 
samples  53  and  54  were  tested  the  Lot 
4538  strengths  and  densities  returned  to 
normal  levels.     The  report  on  the  averages 
for  the  CCRL  samples  will  not  be  available 
until  early  October  and,  therefore,  these 
deviations  are  not  available  in  Figures 
2  and  3. 

Use  of  Standard  Cement  Lot  4538  Data  to 
Correct  Tests  of  Unknown  Cements 

Figure  5  is  a  plot  of  the  Cement  Refer- 
ence Sample  Deviations   (Table  A)  minus  the 
Lot  4538  deviations   (Table  B)   for  batches 
mixed  at  the  same  time.     In  each  instance, 
the  deviations  are  the  average  for  two  batch- 
es or  rounds. 

Figure  1  showed  that  the  strength  levels 
on  the  reference  samples  correlate  well  with 
levels  on  the  Lot  4538  cement  tested  at  the 
same  time.     This  suggests  that  when  unknown 
cements  are  tested  the  results  can  be  cor- 
rected by  the  amount  that  Lot  4538  tests, 
made  at  the  same  time,  are  high  or  low. 

Figure  5  is  a  test  of  this  hypothesis 
and  except  for  a  couple  of  early  results  the 
systematic  variation  of  results  shown  in  the 
control  charts  is  eliminated.     Table  4  pro- 
vides the  averages  and  standard  deviations 
of  these  "corrected"  deviations  along  with 
the  JRL  deviations  in  tests  of  the  CCRL  ref- 
erence samples  and  Lot  4538.     The  period  in- 
cluded is  from  sample  15  through  5  2. 
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The  "corrected  deviations"  average  93 
and  119  psi  at  3  and  7  days,  respectively. 
These  arise  because  during  the  period  the 
average  strength  of  the  Lot  4538  tests  was 
93  and  113  psi  less  than  the  values  used  in 
computing  the  Lot  4538  deviations. 

3  days:   3045  -  93  =  2952  psi 
7  days:  4230  -  119  =  4111  psi 

Therefore/  when  an  unknown  cement  is  tested 
the  best  estimate  of  its  3  day  strength 
would  be : 

Xcorr  =  xv  -   (Xr  "  2952) 

where 

Xcorr  is  the  estimated  3  day  strength 

Xv  is  the  result  of  testing  the  unknow 
cement 

Xr  is  the  strength  of  Lot  4538  mixed  a 
the  s  ame  time . 

(In  each  instance  results  are  the  average  o: 
tests  of  two  batches.) 

The  standard  deviations  of  the  correct< 
values  in  Table  4  are  dramatically  improved 
The  95  percent  confidence  limits  on  tests 
corrected  in  this  manner  should  be  about 
±  250  psi. 

Within  Batch  and  Batch-to-Batch  Surveillanc 


The  control  charts  described  employ  av 
erages  of  2  and  4  batches  of  mortar  and  arej 
designed  to  control  the  troublesome  long 
term  variations.     Control  charts  for  ranges^ 
within  and  between  batches  could  be  main- 
tained but  the  JRL  has  decided  to  use  D2S 
limits  for  the  three  cubes  averaged  for  a 
test  and  for  the  two  or  three  batches  gen-  . 
erally  mixed. 

The  range  of  the  three  cubes  tested  fr 
a  batch  at  a  given  age  in  the  reference  sair 
pie  program  were  extracted  and  expressed  as 
standard  deviation  percent.     The  average  3 
and  7  day  coefficient  of  variations  were 
2.1  and  2.8%,  respecrively .  The  precision 
statement  for  ranges  between  3  cubes  is : 

"The  range  between  3  cubes  made  frc 
a  single  batch  of  C  109  mortar  tested 
at  3  or  7  days  should  not  exceed  7  or 
9  percent  of  their  average,  respec- 
tively  (D2S).     The  corresponding  value 
for  the  range  of  3  cubes  tested  at  28 
days  is  likely  somewhat  larger  than 
that    for  7  day  tests,  but  the  7-day 
values  will  be  used  until  additional 
data  is  developed. [5] " 


No  corresponding  precision  statement  for 
agreement  of  tests  of  3  cubes  is  given  in 
ASTM  Method  C  109. 

The  precision  statement  for  agreement 
of  batches  made  within  a  period  of  several 
days  is  derived  from  the  between  batch  stan- 
dard deviations  in  Table  3.     The  standard 
deviation  chosen  is  3.0  percent  and  is  ap- 
plied for  all  ages.     Note  that  the  C  109 
precision  statement  uses  a  single- labora- 
tory standard  deviation  of  3.8  percent.  The 
JRL  precision  statement  is : 

"The  range  between  batches  (rounds) 
of  C  109  mortar  made  by  a  single  op- 
erator should  not  exceed  the  follow- 
ing at  3  or  7  days :   (Comparable  ranges 
for  tests  at  1  day  are  likely  somewhat 
larger  and  those  at  28  days  somewhat 
smaller. ) 


Range  of  2  rounds 
Range  of  3  rounds 
Range  of  5  rounds 


Range ,  %  of 
Average,  D2S 
8.4% 
9.9% 
11.7% 


Note  that  the  D2S  values  should  be 
exceeded  about  1  time  in  20  in  the 
long  run . " 

Summary 

The  Joint  Research  Laboratory  has  par- 
ticipated in  the  cement  reference  sample 
program  for  the  past  13  years.     For  most  of 
this  time  a  laboratory  standard  cement  has 
been  tested  with  the  reference  samples  and 
when  other  cement  samples  are  tested. 

To  control  the  results  of  ASTM  C  109 
compressive  strength  test  results  the  lab- 
oratory has  developed: 

a.  Difference  2  sigma  limits  for  the 
range  of  the  three  cubes  tested  at 
each  age. 

b.  Difference  2  sigma  limits  for  the 
range  between  batches  (rounds)  mixed 
within  a  short  period. 

c.  Control  charts  for  the  deviation  of 
the  reference  sample  results  from 
the    grand   average  of  all  labora- 
tories on  each  sample. 

d.  Control  charts  of  the  deviation  of 
the  laboratory  standard  cement 
from  its  established  average. 


The  procedure  of  expressing  the  JRL  ref- 
erence sample  results  as  deviations  from  the 
grand  average  for  participating  laboratories 
permits  the  use  of  simple,  familiar,  easily 
understood  statistical  procedures,  calcu- 
lations and  control  charts. 

A  study  of  the  reference  sample  data 
demonstrates  that  in  the  JRL  multiple 
batches  can  be  tested  to  reduce  short  term 
testing  variations  to  the  desired  levels. 
However,  C  109  mortars  tested  at  intervals 
of  6  months  or  more  can  be  expected  to  show 
larger  variations  or  bias  which  will  be 
characteristic  of  the  particular  day  or 
week  on  which  they  are  tested. 

If  the  "standard  cement"  yields  a 
high  result  on  a  given  day,  the  CCRL  refer- 
ence sample  mixed  on  that  day  will  be  sim- 
ilarly high.     Using  the  standard  cement  de- 
viations to  correct  the  CCRL  reference  sam- 
ples mixed  on  the  same  day  significantly 
improves  agreement  with  the  CCRL  averages. 
It  appears  that  if  the  laboratory  standard 
cement  is  mixed  each  day  an  unknown  cement 
sample  is  tested  the  JRL  can  use  the  stan- 
dard cement  results  to  estimate  the  strength 
that  would  be  obtained  if  the  100  to  200 
CCRL  laboratories  had  tested  the  cement. 
The  95  percent  confidence  limit  on  such  an 
estimate  would  be  approximately  ±  250  psi. 
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Table  1. 

Summary  of  JRL  Ratings 

on  Compressive 

Strength 

intiuacu  aire  luc 

ratxngs  ror  .5  ana   /  aay 

strength  of  46 

CCRL  samples  that 

have  been  tested. 

Deviation  from 

JRL, 

Rating 

Av,  Standard 

Expected 

%  of 

No. 

Deviation (1) 

%  of  Labs 

Samples 

5 

0-1 

68.3 

73.9 

4 

1  -  1.5 

18.3 

14.1 

3 

1.5  -  2.0 

8.8 

7.6 

2 

2.0  -  2.5 

3.3 

3.3 

1 

+  2.5 

1.2 

1.1 

(l)Using  CCRL  Multi-lab  Standard  Deviation. 


Table  2.     Typical  JRL  Compressive  Strength  Data 
(Cement  Reference  Sample  Program,  CCRL) 


Test 


Sample 

No.  Operator 

Batch 

Age , 
Days 

C  109  Cube 

Compressive  Strength, 

psi 

1 

2 

3 

Av.  (2) 

51  WDH 

A(l) 

3 

2990 

2990 

2900 

2960 

7 

4415 

4260 

4560 

4410 

B 

3 

3060 

2940 

3040 

3015 

7 

4370 

4435 

4315 

4375 

52  WDH 

A 

3 

3280 

3365 

3480 

3375 

7 

4200 

4200 

4210 

4205 

B 

3 

3435 

3320 

3430 

3395 

7 

4160 

4370 

4350 

4295 

Standard  WDH 

A 

3 

2490 

2540 

2480 

2505 

(Lot  4538) 

7 

3670 

3730 

3770 

3725 

B 

3 

2565 

2625 

2700 

2630 

7 

3635 

3780 

3735 

3715 

(1)  Reported  to  CCRL. 

(2)  Only  the  average  value  for  tests  of  3  cubes  is  used  in  scatter  diagrams 
and  computations  by  CCRL. 
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Table  3.     Summary  of  JRL  Averages  and 
Standard  Deviations  (J-146) 


 Test  Age  

Item    3  Day  7  Day 


Overall  Variations 

1.1  CCRL  Deviations    (From  the  CCRL  Average) 

(to  No.   52)  N  82  84 

X,  psi  -26  +  6 

s,  psi  182  227 

1.2  Lot  4538  Deviations  (From  the  Average  Used  for  Lot  4538) 

(to  No.   54)                           N  38  38 

X",  psi  -78  -74 

s,psi   (%)  208  (7.0)  229  (5.5) 

Average  Strength  psi  2967  4156 

Between  Batch  Std.  Dev., 
psi  (%)    (by  range) 


2.1  Samples  1-40 


2.2  Samples  1-52 


CCRL  95  131 

Lot  4538  114   (3.8)  124  (3.0) 

Av.  104  128 


CCRL  83  HI 

Lot  4538  91_  (3.1)  105  (2.5) 

Av.  87  108 


Table  4.     Correction  of  JRL  Deviations  of  CCRL  Samples 
Using  Lot  4538  Data  —  Samples  15-52 

Data  for  samples  7-12  are  not  included  because  of  difficulties  experienced 
early  in  the  program. 


No. 

Average 

Std.  Dev., 

Item 

Values 

Deviation 

psi 

3 

Day 

Tests 

Deviation 

(a)  CCRL 

30 

-20 

179 

(b)  4538 

15 

-113 

200 

(c)  Corrected 

30 

+93* 

126 

7 

Day 

Tests 

Deviation 

(a)  CCRL 

30 

21 

207 

(b)  4538 

15 

-98 

233 

(c)  Corrected 

30 

+  119* 

121 

*(c)  =  (a)  -  (b) 
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Deviation  of  Lot  4538  from  3045  psi 
at  3  Days  and  4230  psi  at  7  Days 

Figure  1.     Comparison  of  Compressive  Strength 
Deviations  on  Lot  4538  with  Deviations  in  CCRL 
Cement  Reference  Sample  Program 


(Series  J-146) 


CCRL  Sample  No. 
Year 

Figure  2.     Control  Chart  for  3  Day  Strength, 
Cement  Reference  Samples  —  JRL 


(Series  J-146) 
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Figure  4.     Control  Chart  for  Tests  of  (Series  J- 146) 

Cement,  Lot  4538 
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Figure  5.     Reference  Sample  Program  Results       (Series  J-146) 
Corrected  for  Standard  Cement  Results 
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Appendix  A 


Table  A.     Deviation  of  JKL  Results  from  Average  Compressive 
Strength  in  the  Cement  Reference  Sample  Program 

ft  test  is  the  average  of  results  from  3  cubes  from  a  batch  of  ASTM  C  109  mortar.     The  de- 
tion  is  the  difference  between  this  test  result  and  the  average  for  all  participating  labs. 


pie 

3. 


Oper- 
ator 
Date 


3  Day  Strength  Tests 


Deviation,  psi   (JRL-CCRL)  Batch 

Batch   Range , 

A  B  Av.  psi 


7  Day  Strength  Tests 


Deviation,  psi  (JRL-CCRL) 

Batch  

A  B  Av. 


Batch 
Range , 
psi 


ACE 
(8/66) 


ACE 
(3/67) 


WDH 
(8/67) 


WDH 
(2/68) 


20 
150 


144 
69 


■121 
-375 


118 
71 


165 
-66 


-270 
-337 


13 
166 


154 
2 
78 

-196 
■356 
^276 

66 
118 
92 


24 
139 


149 

38 


105 
95 


■157 
12 


34 
■141 


■363 
■711 


-147 
136 


-3 
155 


137 
89 


■365 
■321 


23 
81 


-80 
+84 
+  2 

86 
-26 
30 

-364 
-516 
-440 

-62 
109 
23 


154 
143 


103 
230 


2 

390 


170 

55 


WDH 
(8/68) 


WDH 
(2/69) 


■170 
■166 


■749 
559 


-55 
144 


■1034 
459 


-112 
-11 
-62 

-916 
509 
-203 


115 
310 


235 
100 


80 
7 


■362 
354 


35 
367 


-752 
424 


56 
187 
122 

-557 
389 
-  84 


45 
374 


390 
70 


(8/69) 


WDH 
(2/70) 


279 
96 


19 
6 


149 
51 
100 


260 
90 


286 
33 


51 

238 


168 
136 
152 


235 
205 


JK 
(8/70) 


JK 
(2/71) 


265 
379 


■125 
138 


230 
374 


-165 
13 


248 
376 
312 

-145 
76 
-34 


35 
5 


40 
125 


79 
336 


-198 
247 


229 
331 


-108 
212 


154 
334 
244 

-152 
230 
39 


150 
5 


90 
35 


WDH 
(8/71) 


109 
387 


174 
197 


142 
292 
217 


65 
190 


354 
493 


249 
478 


302 
486 
394 


105 
15 


(con't. ) 
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Table 

a  vcon  u»  j 

3  Day  otrengtn  lests 

7  Day  Strength  Tests 

Oper- 

T~\/-s» » -!      +-  i  f it-i        ricr-i       f  TT3T  —  f^C^TiT  \ 

uevidLion  t  psi    \o  l-v_kxj; 

Da  4-  /-iVi 

Deviation,  psi 

(JKL-CCRL)  B< 

Sample 

ator 

R.3J1CJS  / 

Batch 

No. 

Date 

A                  B  Av. 

psi 

D 

Av.  i 

23 

WDH 

-13              187  87 

200 

170 

405 

ZOO  „ 

24 

(2/  /2) 

1  *3Q                        1  7Q                        1  CO 

x  jy            x  /y  xsy 

182 

312 

O  /t  7 

123 

268 

25 

ZD 

27 

WDH 

-59               41             -  9 

100 

Z  H 

j? 

32 

ZO 

\£/  1  -5  ) 

Z  Z                        1  X*± 

-73 

37 

-18 

-12 

7 

Z 

Z 

-49 

30 

(8/73) 

84 

131 

**  1 
J  JL 

1J7 

47? 

*X  1  Z 

32 

(2/74) 

-302 

947 

Z  •*  1 

0  J) 

1  /I 

35 

WDH 

-359            -319  -339 

40 

-287 

-392 

-340 

Jo 

.CO                        _1  _QQ 

DO                          lJO                             V  O 

Z  o 

zUo 

—  TIB 

xxo 

z  xo 

-229 

37 

WDH 

-116                54  -31 

170 

Its  / 

37 

J  O 

lb/  /  D  J 

.3U*i           ~xy*±  zfiy 

Tin 

-280 

-105 

-192 

7Q 

-  /O 

WDH 

o4                    J4  by 

bU 

268 

103 

186 

40 

(2/76) 

103                28  _66_ 

75 

101 

31 

DO 

o  z 

JLZu 

41 

WDH 

57                37  47 

20 

-64 

56 

—  A 

a  n 
4z 

/                 -oJ  ZO 

7n 

74 

159 

1  1  C 
XXO 

xu 

~56 

43 

WDH 

-7                28  10 

35 

-81 

-31 

_ 

3D 

\4/  II) 

JO                         JU                         J  Z 

C 

z> 

iy 

Q  A 

CO 
Z)Z 

XX 

"2 

45 

WDH 

-136            -126  -131 

10 

-66 

-96 

-81 

la  inn \ 
1°/  1  1 1 

— yz             —  yz               y  z 

u 

-139 

-74 

-106 

94 

/J  1 

WDH 

-24             -234  -129 

210 

-65 

-75 

-70 

48 

(2/78) 

-172             -167  -170 

5 

-158 

-203 

-180 

-149 

-125 

(con't. 
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Table  A  (con" t. ) 


3  Day  Strength  Tests 


Oper-  Deviation,  psi  (JRL-CCRL) 

Sample        ator  Batch 
No.  Date  A  B  Av. 


7  Day  Strength  Tests 


Batch 
Range , 
psi 


Deviation,  psi  (JRL-CCRL) 
Batch 

A  B  Av. 


Batch 
Range , 
psi 


49 
50 


WDH 
(8/78) 


■179 
■258 


-94 
-48 


-136 
-153 
-144 


85 
210 


■215 
■221 


-65 
-156 


-140 
-188 
-164 


150 
65 


51 
52 


WDH 
(2/79) 


-486 
■166 


-431 
-146 


-458 
-156 
-307 


55 
20 


•387 
-210 


-422 
-120 


-404 
-165 
-284 


35 
90 


53 
54 


WDH 
(8/79) 


365 
195 


65 
200 


Av.  Range,  R 

1-40 

1-54 


108 
94 


148 
125 


*Sample  numbers  were  probably  interchanged.    Excluded  from  calculations. 
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Appendix 

B 

Table 

B.     Deviation  of  Lot  4538  Tests 

from  Established  Average  Values 

Tests  of  C  109  mortars  made  with  carefully  preserved  samples  of  cement,  Lot  4538.  Values 
are  expressed  as  the  test  value  minus  3045  psi  at  3  days  and  the  test  value  minus  4230  psi 
at  7  days. 

3  Day  Strength  Tests 

7  Day  Strength  Tests 

Oper- 

Deviation,  psi  (X-3045) 

Batch           Deviation,  psi   (X-4230)  Bat 

Sample  ator 

Batch 

Range ,               Batch  Ran 

No. 

Date 

A                B  Av. 

psi               A                B               Av.  ps 

1 

3 

5 

7 

WT1H 

(2/68) 

240              220  230 

20              135              275              205  1A 

Q 

wun. 
(8/68) 

95              145  120 

50             -15               35               10  E 

1 1 

WDH 
(2/69) 

160            -225  -32 

385                45            -295            -125  3< 

15 

WDH 
(2/70) 

210              -20  95 

230              250              -80                85  3 

17 

JK 
(8/70) 

65              195  130 

260                15              250              132  2 

19 

JK 
(2/71) 

-105            -115  -110 

10              -90              -25  -58 

21 

WDH 
(8/71) 

240              215  228 

25              450              275              362  1 

23 

WDH 
(2/72) 

-60                  0  -30 

60                15                65  40 

25 

27 

WDH 
(2/73) 

45            -110  -32 

155                 0              -85  -42 

29 

WDH 

31 

WDH 

(con't.) 

122 

.  IB 

Table  B  (con't.) 


3  Day  Strength  Tests 


Oper- 
Sample  ator 
No.  Date 


Deviation,  psi  (X-3045) 
Batch 

Av. 


7  Day  Strength  Tests 


A 


B 


Batch 
Range , 
psi 


Deviation,  psi  (X-4230)  Batch 

Batch   Range , 

A  13  Ay.  psi 


33 
35 


WDH 
(2/75) 


■430 


-410 


-420 


20 


-420 


-405 


-412 


15 


WDH 
(8/75) 


-25  -205  -115 


180 


-175 


-250 


-212 


75 


39 


WDH 
(2/76) 


-40 


-190  -115 


150 


40 


-85 


-22 


125 


41 


WDH 
(8/76) 


-40 


-90 


-65 


50 


-40 


-125 


-82 


85 


43 


WDH 
(2/77) 


-70 


-30 


-50 


40 


-40 


35 


-  2 


75 


45 


WDH 
(8/77) 


-75 


-80 


-78 


-45 


-35 


-40 


10 


47 


WDH 
(2/78) 


-395 


-295 


■345 


100 


-395 


-225 


-310 


170 


49 


WDH 
(8/78) 


■355 


■275 


■315 


80 


-400 


-405 


-402 


51 


WDH 
(2/79) 


■540 


-415  -478 


125 


-505 


■515 


■510 


10 


53 


WDH 
(8/79) 


-95 


-95 


-95 


-125 


85 


-20 


210 


Av .  Range ,  R 

::  1-40 

1-53 


129 
102 


140 
118 
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Formal  standardized  procedures  for  thermal  performance  tests  were 
established  at  the  Johns-Manville  R&D  Center  as  part  of  a  program  to  assure 
reliability  of  test  data.     A  number  of  requirements  had  to  be  met  by  the 
procedures.     While  the  tests  are  based  on  ASTM  and  other  similar  recognized 
standard  methods,  the  procedures  had  to  be  sufficiently  detailed  to  insure 
consistently  reliable  results  by  all  operators,  some  of  whom  may  not  have 
had  long-term  experience  in  thermal  testing.     Likewise,  the  procedures  had 
to  cover  a  wide  variety  of  types  and  forms  of  materials,  test  equipment 
and  temperature  ranges.     A  complicating  factor  was  the  inter-relation 
between  general  lab  procedures  applicable  to  all  activities,  procedures 
for  checking  general  purpose  equipment  and  specific  thermal  test  equip- 
ment, thermal  test  methods,  and  test  specimen  preparation  procedures  for 
specific  types  of  materials. 

This  paper  describes  the  formats  for  a  series  of  formal  cross- 
referenced  standardized  procedures  for  thermal  performance  tests.  The 
paper  also  details  some  of  our  experiences  in  preparing  and  implementing 
these  test  procedures. 

Key  words:     Laboratory  test  procedures;  quality  assurance  lab  test; 
standard  test  methods;  thermal  performance  test. 


1.  Introduction 

The  basic  objective  of  a  quality  assur- 
ance program  for  a  testing  laboratory  is  to 
insure  the  consistent  reliability  of  the  test 
results  obtained.     This  was  also  the  objec- 
tive of  the  program  at  the  Thermal  Conduc- 
tivity Laboratory,  or  K-LAB,  at  the  Johns- 
Manville  Research  and  Development  Center  in 
Denver,  Colorado. 


under  development,  and  competitive  product 
In  addition,  a  limited  amount  of  outside  o 
contract  testing  is  performed.  Thermal  te 
procedures  involve  establishing  steady-sta 
heat  flow  under  prescribed  conditions,  and 
then  measuring  the  heat  flux  and  the  surfa 
temperatures.  Related  properties,  such  as 
physical  dimensions  and  density,  are  also 
determined  because  of  their  influence  on 
thermal  performance. 


The  K-LAB  performs  a  wide  variety  of 
thermal  performance  tests.     The  scope  in- 
cludes various  types  and  forms  of  materials, 
which  may  be  tested  individually  or  in  com- 
bination with  other  materials  and  at  various 
temperatures.     Samples  submitted  for  test  in- 
clude materials  from  regular  plant  production 
for  routine  quality  control,  new  products 
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The  K-LAB  has  long  operated  under  in 
formal  standard  thermal  test  procedures. 
Many  of  these  served  as  the  basis  for  star 
dard  test  methods,  subsequently  published 
ASTM,  ANSI,  or  ISO  standards.     Formerly,  1 
testing  quality  assurance  relied  on  the  si 
and  long-term  experience  of  the  K-LAB  ope] 
tors  with  their  testing  equipment.  While 


:':    test  procedures  generally  followed  the  re- 
'>i    quirements  of  the  published  standard  test 
methods,  there  were  many  minor  deviations 
where  experience  had  demonstrated  published 
test  criteria  were  either  too  lax  or  overly 
restrictive,  or  special  needs  had  to  be  met. 

The  incentive  for  the  J-M  K-LAB  to  es- 
tablish formal  test  procedures  came  from 
several  directions .     There  was  need  for  fur- 
ther improvements  in  precision  and  accuracy. 
The  volume  of  testing  had  increased.  Labo- 
ratory personnel  have  been  added,  promoted, 
|  and  transferred,  resulting  in  a  transient 
!  personnel  situation.     The  requirements  of 
j  the  national  and  state  thermal  test  labora- 
tory certification  and  accreditation  programs 
were  another  factor.     An  internal  requirement 
in  establishing  the  formal  K-LAB  standard 
procedures  was  that  they  be  compatible  with 
j  other  J-M  and  R&D  systems  already  in  exis- 
tence.    For  example:     The  R&D  Information 
Center  had  previously  established  procedures 
to  be  followed  in  recording  data  in  perma- 
nent lab  notebooks,  and  also  had  general 
|   formats  for  reports.     Thus,  the  K-LAB  pro- 
cedures had  to  complement  the  other  systems. 

2.     Standard  Procedures 

Thermal  performance  testing  was  analyzed 
carefully  to  identify  areas  where  standard 
procedures  were  needed  to  meet  testing  quali- 
ty assurance  requirements.     The  task  was  to 
separate  common  activities  of  the  various 
steps  in  testing,  and  to  rearrange  them  in 
logical  combinations  that  would  lead  to  the 
preparation  of  cohesive  standard  procedures. 
For  example,  many  of  the  details  of  labora- 
tory operation  apply  to  all  activities  irre- 
spective of  the  complexity  of  the  test  pro- 
cedure involved  or  the  type  of  material  be- 
ing tested.     Certain  pieces  of  laboratory 
equipment,   such  as  temperature  indicators 
j  and  recorders,  are  in  general  use  rather 
•:  than  being  associated  with  a  particular 

piece  of  thermal  test  equipment.  Further, 
•:  except  for  physical  size  consideration,  the 
.  :  steps  involved  in  the  preparation  of  speci- 
.    mens  for  testing  is  more  a  function  of  the 
character  of  the  material  being  tested  than 
i  the  particular  piece  of  thermal  test  equip- 
ment that  will  be  utilized. 

What  evolved  was  a  plan  for  six  classes 
of  standard  procedures,  with  the  recognition 
that  extensive  use  of  cross  referencing 
would  be  utilized.     These  are  enumerated  in 
Table  0. 


Table  0  -  Classification  of  Thermal 
Test  Procedures 

1.  General  lab  operation 

2.  Check  adjunct  apparatus 

3.  Check  thermal  test  apparatus 
it.  Thermal  test  methods 

5.  Test  specimen  preparation 

6.  Standard  reference  check  specimens 

The  first  area  covers  the  general  oper- 
ation of  the  K-LAB,  including  as  standard 
procedures,  authorization  of  tests,  handling 
of  samples  received,  retention  of  data  and 
information,  and  reporting  of  test  results. 
The  nature  of  thermal  testing  requires  the 
use  of  both  common  or  general  test  equipment 
that  might  be  used  in  several  test  proce- 
dures, and  specific  test  equipment  for  only 
one  type  of  test.     Checking  procedures  for 
general  or  adjunct  test  equipment  are  cov- 
ered in  the  second  area,  while  specific  test 
apparatus  are  in  the  third  area.     The  actual 
testing  procedure  to  be  followed  in  conduct- 
ing a  specific  test  is  covered  in  the  fourth 
area.     Procedures  for  the  preparation  of 
specimens  for  the  test  are  covered  in  the 
fifth  area.     This  is  divided  by  types  of 
materials,  i.e.,  batts  and  blankets,  rigid 
board  and  block,  pipe  insulation,  and  re- 
fractory materials.     In  the  sixth  area,  the 
preparation  and  handling  of  standard  refer- 
ence check  specimens  is  covered. 

The  decimal  system  was  utilized  to 
classify  types  of  procedures  within  a  class, 
and  sections  within  a  particular  procedure. 

Thus,  as  shown  in  Table  0,  the  number 
h  identifies  a  procedure  as  a  thermal  test 
method.     The  number  it.  3  is  a  thermal  test 
method  for  a  specific  piece  of  test  appara- 
tus, in  this  case,  the  high  temperature 
calorimeter  complying  with  ASTM-C182;  the 
number  it.  3.7  is  a  section  within  that  test 
method. 

Details  of  the  formats  used  in  each  of 
the  areas  follow. 

3 .     Format  s 

The  first  area  of  standard  procedures 
deals  with  laboratory  operation.     This  has 
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"been  divided  into  two  procedures;  1.1  covers 
the  general  mechanics  of  the  K-LAB  operation, 
while  1.2  details  reports.     Because  of  the 
importance  of  the  proper  written  communica- 
tion of  technical  data  by  testing  laboratory, 
reporting  procedures  were  put  into  a  separate 
procedure.     In  this  procedure,  general  re- 
quirements to  be  met  by  all  reports,  in  ad- 
dition to  specific  formats  for  the  five 
types  of  reports  issued  by  the  K-LAB,  are 
covered.     Table  1  details  the  format  for  1.1 
and  1.2  procedures. 

Table  1  -  Format  -  Laboratory 
Operation  Procedures 

1.1  General  K-LAB  procedures 

1.1.1  Scope  -  applicability,  categories  of 
procedure 

1.1.2  Hazards  -  general  caution 

1.1.3  Index  -  procedures 
1.1.1+    Test  Authorization 

1.1.5  Assignment  -  assignor,  responsibil- 
ity 

1.1.6  Schedule  -  priority  of  tests 

1.1. T    Test  Data  -  reference  to  permanent 
notebook  records 

1.1.8  Report  -  reference  to  1.2 

1.1.9  Sample  disposition  -  retention 

1.1.10    K-LAB  information  -  types  of  records, 
retention 

1.2  K-LAB  reports 

1.2.1  Scope  -  all  written  communication  of 
thermal  performance  data 

1.2.2  Types  -  description,  distribution 
limits 

1.2.3  Style  -  no  set  writing  style,  units 
(US  vs.  metric) 

1.2.1+    General  requirements  -  applicable 
all  reports 

1.2.5  Format  -  test  report 

1.2.6  Format  -  development  report 

1.2.7  Format  -  external  report 


1.2.8  Format  -  memo  report 

1.2.9  Format  -  letter  report 

An  essential  part  of  the  quality  assurl 
ance  program  of  any  testing  laboratory  is 
the  periodic  checking  and  calibration  of 
testing  equipment  used.  Calibration  record! 
of  all  testing  equipment  in  the  Material 
Performance  Section  or  physical  testing 
group  at  the  J-M  R&D  Center,  which  includes1 
the  K-LAB,  have  been  put  on  computer.  A 
typical  print-out,  (Figure  l),  describes  th'l 
equipment  including  its  range,  usual  loca-  j 
tion  and  person  responsible.  Calibration 
information  includes  who  usually  performs 
the  work,  recommended  frequency,  dates  of 
past  calibrations  and  the  next  due  date.  Jj 
periodic  review  of  the  computer  files,  attn 
tion  can  be  focused  on  equipment  past  its  | 
normal  check  or  calibration  date. 

Standard  pieces  of  laboratory  equip- 
ment, such  as  scales  and  balances,  are  ser-l 
viced  routinely  by  an  outside  contractor, 
frequently  the  manufacturer's  local  repre- 
sentative.    No  attempt  is  made  to  dictate 
checking  procedures  for  this  equipment. 
Primary  electrical  and  temperature  standar 
and  measuring  equipment  are  sent  out  peri- I 
odically  to  a  qualified  local  metrology  laj 
oratory  with  standards  traceable  to  NBS. 
Again,  no  attempt  is  made  to  dictate  pro- 
cedures followed.     Only  for  that  test  equi: 
ment  which  is  checked  and  calibrated  by 
K-LAB  staff  have  standard  procedures  been 
written.     Those  pertaining  to  adjunct  test] 
apparatus  are  covered  by  the  2.x  series  of 
procedures  (see  Table  2  for  format  details 
Similarly  the  procedures  for  checking  therj 
mal  test  apparatus  are  in  the  3.x  series 
(see  Table  3) . 

Table  2  -  Format  -  Adjunct  apparatus 
check 

2.x. 1    Scope  -  type(s)  apparatus  covered 

2.x. 2    Hazards  -  general  caution,  and  spe- 
cific hazards  (if  any) 

2.x. 3    Apparatus  -  description  of  apparati 
checked 

2.x. h    Frequency  -  frequency  of  checks 

2.x. 5    Test  equipment  -  list  of  test  equi 
ment 

2.x. 6    Procedure  -  detailed  check/calibra 
tion  procedure,  precautions,  recorj 
explanatory  drawings 
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Table  3  -  Format  -  Thermal  test 
apparatus  check 

3.x. 1    Scope  -  type(s)  of  thermal  test 
apparatus  covered 

3.x. 2    Hazards  -  general  caution,  and 
specific  hazards  (if  any) 

3.x. 3    Standard  Test  -  reference  to  standard 
test  method,  such  as  ASTM,  and  list 
of  exceptions  (if  any) 

3.x. h    Apparatus  -  description  of  apparatus 
checked 

3.x. 5    Frequency  -  frequency  of  checks  (may 
he  various  levels  of  detail) 

3.x. 6    Log  -  permanent  records  to  he  kept 

3.  x. 7    Procedure  -  test  equipment  and  stan- 

dards required,  detailed  check/cali- 
bration procedure,  precautions, 
explanatory  drawings  (daily  checks 
of  thermal  test  equipment  are  cov- 
ered as  part  of  area  h  on  thermal 
test  methods ) . 

The  procedures  to  he  followed  in  actu- 
ally conducting  a  thermal  performance  test 
are  described  in  the  h.x  series.     When  the 
program  is  completed  a  separate  procedure 
will  have  been  prepared  for  each  type  of 
thermal  test  apparatus.     Details  of  the  for- 
mat for  4.x  procedures  are  shown  in  Table  h. 

Table  h  -  Format  -  Thermal  test 
methods 

4.  x. 1    Scope  -  type(s)  apparatus  covered, 

materials  typically  tested,  tempera- 
ture range  and  size  and  other  limits. 

h.x. 2    Hazards  -  general  caution,  and  spe- 
fic  hazards  (if  any) 

H.x. 3  Standard  test  -  reference  to  stan- 
dard test  method  such  as  ASTM,  and 
list  of  exceptions  (if  any) 

4.x.  4    Specimen  -  test  specimen  description, 
and  reference  to  standard  procedures 
5.x  on  preparation 

k.x. 5    Apparatus  and  equipment  -  list  of 
test  equipment  required 

h.x. 6    Procedure  -  Initial  startup,  daily 
checks  of  apparatus,  detailed  test 
procedure,  precautions,  data  require- 
ments (including  examples)  and  daily 
shut  down 


h.x. 7    Calculations  -  equations,  procedure, 
hand  calculator  program  (if  used) 
and  example 

U.x. 8    Figures  -  explanatory  drawings 

Procedures  describing  the  preparation 
of  thermal  test  specimens  are  divided  by 
type  of  material  to  be  tested,  with  separate 
procedures  for  batt /blanket ,  loose- fill,  and 
rigid  board  and  blocks.     Because  of  special 
requirements,  separate  procedures  were  also 
prepared  for  pipe  insulation  and  for  refrac- 
tory materials.     Details  of  the  format  5.x 
procedures  for  test  specimen  preparation  are 
included  in  Table  5- 

Table  5  -  Format  -  Test  specimen 
preparation 

5.X.1    Scope  -  type  of  material  covered 

5.x. 2    Hazards  -  general  caution,  and  speci- 
fic hazards  (if  any) 

5.x. 3    Standard  test  -  reference  to  standard 
test  methods,  such  as  ASTM,  and  list 
of  exceptions  (if  any) 

5.x. k    Apparatus  and  equipment  -  list  of  all 
equipment  required 

5.x. 5    Procedure  -  selection,  I.D.,  cutting 
to  size,  drying  and  other  pre-test 
conditioning,  physical  measurements, 
data  requirements,  and  precautions 

5.x. 6    Calculations  -  equations,  procedure, 
hand  calculator  program  (if  used), 
and  example 

5.x. 7    Figures  -  explanatory  drawings 

Preparation  of  procedures  for  the  last 
area,  the  preparation  and  checking  of  stan- 
dard reference  test  specimens,  has  been 
held  in  abeyance.     An  increasingly  important 
portion  of  thermal  performance  testing  re- 
lies on  heat  flow  transducers  to  determine 
thermal  flux.     In  the  past,  NBS  has  made 
available  a  moderately  heavy  density  glass 
fiber  board  in  1  inch  thickness  only,  as  a 
standard  thermal  reference  material  (SRM). 
This  standard  was  adequate  for  calibration 
of  heat  flow  transducers  when  thermal  test- 
ing accuracy  requirements  were  far  less 
stringent  than  today.     Current  technology- 
has  shown  that  many  discrepancies  can  occur 
when  an  apparatus  calibration  based  on  the 
current  NBS  thermal  SRM  is  extended  to  ma- 
terials of  different  characteristics  than 
the  standard,  and  especially  when  extended 
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to  the  greater  thicknesses  common  in  today's 
insulation  standards. 

NBS  now  has  a  program  under  way  to  de- 
velop equipment  which  will  permit  them  to 
offer  a  wider  variety  of  thermal  SRM's.  Out- 
need  for  6.x  procedures  will  he  inversely 
related  to  the  success  and  timing  of  the  NBS 
program. 

k.  Observations 

The  preparation  and  implementation  of 
standard  thermal  performance  test  procedures 
has  required  a  large  investment  in  time  on 
the  part  of  the  personnel  involved.  This 
was  not  unexpected,  as  the  need  for  accuracy 
and  completeness  of  presentation  was  recog- 
nized early.     Limited  availability  of  time 
necessitated  establishing  priorities,  so 
that  the  more  important  procedures  were 
attacked  first.     K-LAB  personnel  had  to  con- 
tinue regular  testing  activities  at  the  same 
time . 

The  question  of  how  much  detail  to  in- 
clude in  a  standard  procedure  requires 
judgement.     A  fine  balance  is  required  be- 
tween providing  sufficient  detail  to  cover 
all  requirements,  and  overloading  a  proce- 
dure with  the  "obvious."     One  device  which 
worked  well  in  the  preparation  of  an  initial 
draft,  was  for  an  experienced  operator  to 
use  a  pocket  dictating  machine  to  record 
every  step  he  followed  as  he  went  through 
a  test.     This  was  edited  to  insure  essential 
details  were  covered  and  non-essential  ones 
eliminated. 


all  operators  having  standard  procedures 
available.     A  perhaps  not  so  obvious  benefit 
was  the  critical  analysis  of  each  step  dur- 
ing the  preparation  of  a  standard  procedure. 
The  following  questions  were  asked  many 
times:     Is  this  step  really  necessary?  Is 
there  a  better  way  in  terms  of  improved 
accuracy  or  reduced  operator  time? 

We  plan  continued  efforts  to  complete 
the  balance  of  the  standard  procedures.  In 
the  future,  we  also  plan  periodic  review, 
to  insure  that  the  standard  procedures  rep- 
resent the  best  methods  for  achieving  reli- 
able test  results  and  that  the  operators  are 
following  the  methods  established. 


The  author  is  deeply  appreciative  of  the 
K-LAB  staff  for  their  helpful  comments  on 
the  general  plan,  and  for  their  assistance 
in  preparing  most  of  the  actual  procedures 
Without  their  extra  efforts  the  program 
could  not  have  reached  its  present  stage. 


You  will  note  reference  in  each  proce- 
dure to  the  hazards  involved.    While  not 
particularly  dangerous  as  such,  thermal 
testing  does  involve  possible  personal  con- 
tact with  elevated  temperature  surfaces, 
high  voltages,  dusts,  and  cutting  devices 
such  as  a  band  saw.     Over  the  years,  the 
R&D  Center  has  maintained  an  enviable  safety 
record  within  Johns-Manville .    This  has  been 
achieved  only  by  constant  attention  to  the 
hazards  involved  and  careful  work  habits  on 
the  part  of  the  personnel. 

The  availability  of  computer-controlled 
drafting  facilities  expedited  the  prepara- 
tion of  explanatory  diagrams.     These  were 
prepared  quickly,  were  of  excellent  clarity, 
and  in  contrast  with  photographs  showed  only 
those  details  considered  pertinent.  Figure 
2  is  an  example  of  a  computer  drafted  dia- 
gram. 

The  obvious  benefit  of  improved  testing 
precision  and  accuracy  has  been  realized  by 
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Figure  1  -  Example  of  Computer  Calibration  Record 
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LABORATORY  PERFORMANCE  EVALUATION 
A  NEW  LOOK  AT  QUALITY  ASSURANCE  IN 
THE  TESTING  LABORATORY 

DONALD  J.  MC  CLAIM 

QUALITY  ASSURANCE  STAFF,  BARTON  INSTRUMENTS 
A  UNIT  OF  INTERNATIONAL  TELEPHONE  AND  TELEGRAPH  CORP. 
CITY  OF  INDUSTRY,  CALIFORNIA  91016 


Safety  and  liability  lavs,  consumer  awareness  and  Federal  laws 
governing  test  requirements  necessary  to  qualify  a  component  for  use  in 
Nuclear  reactors  has  caused  a  new  look  at  laboratory  performance.  The 
independent  laboratory  is  preferred  by  legal  departments  since  it  avoids 
the  connotation  of  conflict  of  interest  in  any  litigation  which  might 
result  if  a  component  fails.     Quality  Assurance  organizations  have  re- 
vised their  methods  of  audits  and  approach  to  the  control  requirements 
needed  by  independent  test  laboratories.     This  reexamination  has  shown 
that  in  many  cases  laboratory  performance  has  not  been  what  the  labor- 
atory customer  expected.     This  paper  presents  not  only  the  requirements 
for  nuclear  qualification  but  assesses  those  performance  standards  cur- 
rently found  in  test  laboratories.     Further  and  most  important,  it  pre- 
sents several  solutions  to  the  problems.     These  are  solutions  that  have 
been  used  successfully.     They  require,  as  most  human  endeavors,  under- 
standing and  cooperation  as  well  as  acceptance  of  certain  responsibilities 
on  both  the  laboratory  and  the  laboratory  customer. 

Key  words:     Advent  of  safety,  liability  and  Federal  Nuclear  laws  requiring 
tests;  consumer  awareness  and  legality;  customer  and  laboratory  responsi- 
bilities; inadequacies  of  test  laboratories;  Institute  of  Electrical  and 
Electronic  Engineers  Specifications;  laboratory  capabilities;  laboratory 
performance  evaluation;  manufacturers  response  to  laws  and  consumers;  one 
way  to  solve  the  problem;  pretest  quality  planning;  tests  required  for 
nuclear  or  safety  related  products;  what  the  laboratory  customer  expects 
in  a  test. 

1.  Introduction 

1.1 

The  advent  of  the  safety  and  liability 
laws  and  Federal  laws  governing  safety  re- 
lated products  for  Nuclear  applications  has 
caused  Quality  Assurance  organizations  to 
revise  quality  programs  in  the  area  of  qual- 
ification testing  of  products. 

1.1.1 

Included  in  these  laws  is  Title  10, 
Code  of  Federal  Regulations,  Part  50, 
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appendix  B,  commonly  referred  to  as  10CFR50, 
which  is  applicable  to  all  activities  af- 
fecting the  safety  related  functions  of 
nuclear  power  plants  and  describes  the  qual- 
ity program  requirements  for  design,  con- 
struction, and  operation  of  these  structures, 
systems  and  components.     In  addition,  ANSI 
Ni+5.2  was  prepared  for  general  industry  use 
as  a  guide,  for  implementation  of  10CFR50, 
appendix  B.     The  design  control  requirements 
of  these  documents  includes  methods  of  de- 
sign verification,  one  of  which  is  qualifi- 
cation testing. 


1.2 

Cursory  discussions  with  legal  experts 
leads  to  the  conclusion  that  qualification 
tests  by  an  independent  test  laboratory  is 
preferred,  since  it  precludes  allegations  in 
litigation  regarding  conflict  of  interest  in 
test  results.     Therefore,  the  independent 
laboratory  performance  becomes  extremely 
important . 

1.3 

Qualification  testing  of  safety  related 
products  is  specified  by  the  Institute  of 
Electrical  and  Electronic  Engineers  specifi- 
cations, IEEE- 323  and  3hk. 

1.3.1 

In  addition,  revised  laws  for  product 
safety  and  liability  have  resulted  from  con- 
sumer awareness  and  have  necessitated  changes 
in  manufacturers '  thinking  regarding  product 
qualification. 

1.3.1.2 

As  a  result,  qualification  testing  of 
new  products,  review  and  possible  requalifi- 
cation  of  existing  products  has  become  man- 
datory in  many  company  operating  policies. 

1.3.1.3 

Such  has  been  the  case  at  ITT  Barton 
whose  written  policy  is  to  perform  qualifi- 
cation testing  on  every  product  we  sell. 

1.3.1. it 

In  addition,  the  testing  imposed  by  the 
Institute  of  Electrical  and  Electronic  Engi- 
neers specifications  IEEE-323  for  safety  re- 
lated products  such  as  instruments  used  in 
Nuclear  power  plants  is  stringent.     I  will 
briefly  discuss  these  tests  in  order  that 
you  will  be  more  able  to  appreciate  the  se- 
rious nature  of  what  I  have  to  say  relative 
to  the  subject  of  laboratory  performance. 

2.     Test  Classification 

2.1 

The  IEEE-323  specification  defines  a 
Class  IE  safety  related  product  as,  "The 
safety  classification  of  the  electric  equip- 
ment and  systems  that  are  essential  to  emer- 
gency reactor  shutdown,  containment  isola- 
tion reactor  core  cooling  and  containment 
and  reactor  heat  removal  or  otherwise  are 


essential  in  preventing  release  of  radio- 
active material  to  the  environment." 

2.1.1 

Each  type  of  Class  IE  power  equipment 
must  be  qualified  by  analysis,  successful 
use  under  similar  conditions,  or  by  actual 
test  to  demonstrate  its  ability  to  perform 
its  function  under  normal  and  Design  Basis 
Events  (DBE's). 

2.1.2 

Design  Basis  Events  are  earthquakes, 
high  radiation  levels  and  loss  of  coolant 
accidents . 

2.1.3 

Most  manufacturers  of  Class  IE  safety 
related  products  qualify  by  a  combination  c 
analysis  and  testing. 

3.     Test  Requirements 

3.1 

Specification  IEEE-323  requires  ageing 
or  simulated  end  of  life  test  typically  10 
to  kO  years,  as  the  first  of  a  series  of 
tests  that  must  be  performed  sequentially. 
ITT  has  found  that  very  few  laboratories 
have  this  capability  or  specialized  knowl- 
edge . 

3.1.2  Ageing 
3.1.2.1 

The  ageing  requirement  exists  so  that 
by  test  it  can  be  proven  that  an  instrumen 
to  be  used  in  nuclear  applications  can  ope 
ate  within  design  parameters  and  operation 
accuracies  for  39  years,  36k  days,  23  hour 
and  59  minutes  and  then  suffer  a  Design 
Basis  Event  such  as  an  earthquake  or  loss 
of  coolant  accident  and  the  instrument  wil 
still  operate  during  and  after  the  Design 
Basis  Event. 

3.1.3  Radiation 


It  follows,  therefore,  after  completi 
of  ageing  tests  the  instrument  is  subjecte 
to  the  next  IEEE-323  test  of  Gamma  Radia-  | 
tion  at  the  level  of  200  x  10°  Rads.  For 
comparison,  the  human  body  limit  of  Gamma 
radiation  if  hOO  Rem.  The  radiation  tests 
insure  that  materials  such  as  wiring  and 
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electronic  components  will  not  fail  due  to 
accumulative  Gamma  radiation  absorbed  over 
the  life  of  the  instrument  and  therefore, 
will  still  perform  the  intended  function  at 
end  of  life  during  a  Design  Basis  Event. 

3.1.*+  Seismic 

3.1.U.1 

The  next  test  in  sequence  is  the  Seis- 
mic simulation  test  specified  in  IEEE- 3^ 
and  is  conducted  on  a  biaxial  seismic  simu- 
lator.    Testing  consists  of  5  operational 
basic  earthquake  or  OBE  tests  of  approxi- 
mately 35  seconds  during  which  the  instru- 
ment operation  and  outputs  are  remotely  re- 
corded.    These  OBE  tests  are  followed  by  one 
SSE  or  Safe  Shutdown  Earthquake  test  of 
approximately  35  seconds  but  at  higher  re- 
sponse acceleration  in  peak  G's.  Frequen- 
cies are  enveloped  from  1  Hz  to  100  Hz  at 
nominal  peak  response  acceleration  of  2  to 
15  G's  and  are  conducted  in  both  horizontal 
and  vertical  axes. 

3.1-5    Loss  of  Coolant  Accident 

3.1.5-1 

The  last  test  is  again  contained  in 
IEEE  specification  323  and  is  called  the 
LOCA  or  Loss  of  Coolant  Accident  test.  Here 
the  specimen  is  subjected  to  saturated  steam 
at  320°F  and  left  to  operate  while  recording 
outputs  and  operational  functions  hopefully 
within  design  limits  for  as  much  as  100  days 
The  specimen  under  test  is  sprayed  with  a 
caustic  solution  during  the  1st  2k  hours  of 
LOCA  test. 

3.1.6    Test  Laboratory  Performance 

3.1.6.1 

Although  we  have  had  many  successful 
tests  at  such  laboratories,  we  have  also 
experienced  such  problems  as  inadequate 
test  planning,  test  set-ups  that  fail  during 
test,  inadequate  audit  of  test  functions  by 
laboratory  personnel,  missing  or  erroneous 
log  keeping  and  data  recording  and  even  the 
destruction  of  our  test  specimen.  ITT 
Barton  and  other  quality  assurance  organi- 
zations have  reexamined  test  laboratory  re- 
lationships with  prospective  customers  by 
improving  communications  through  mutual 
understanding  of  what  is  expected  during 
the  conduct  of  a  test.  Responsibilities 
are  incumbent  on  both  parties.  Although 
there  are  several  solutions,  an  approach 
which  has  been  successful  is  pre-test  qual- 
ity planning. 


3.1.6.2    Pre-test  Quality  Planning 
3.1.6.3 

A  joint  meeting,  prior  to  entering  into 
any  contract,  which  must  include  technical, 
purchasing/contracts  and  quality  assurance 
personnel  during  which  the  following  sub- 
jects, at  a  minimum  must  be  discussed: 

-  Referenced  quality  program 
requirements . 

-  Errors  and  omissions  in  pro- 
posed p.o.  and  specifications. 

-  The  technical  approach  to 
test  methods. 

-  Instrumentation  requirements. 

-  Test  set-up  adequacy. 

-  Test  set-up  check-out  prior 
to  test  start. 

-  Test  process  checklists. 

-  Test  deviations. 

-  Test  data  recording  and  re- 
tention . 

-  Test  reports. 

3. 1.6. it 

Contractor  responsibility  must  embrace 
the  philosophy  to  assist  the  laboratory  in 
achieving  timely  and  specified  performance 
results . 

3.1. T    Laboratory  Performance 
Requirements 

3.1.7.1 

The  laboratory  customer  is  purchasing 
expertise,  not  just  facilities  and  test  per- 
sonnel assistance.     The  experience  of  the 
laboratory  can  do  much  to  prevent  aborting 
tests,  assuring  required  test  conformance, 
reducing  costs,  and  improving  laboratory 
competitiveness . 

3.1.7.2 

Instrumentation  and  test  set-ups  must 
provide  the  intended  results.     Assurance  of 
this  results  from  set-up  check-out,  prior  to 
notification  to  contractors  that  the  labor- 
atory is  ready.     Such  action  insures  mini- 
mum costs  to  both  laboratory  and  contrac- 
tors . 

3.1.7.3 

In  complex  tests  such  as  those  refer- 
enced in  IEEE-323,  which  include  simulated 
life  ageing,  gamma  radiation,  seismic  and 
loss  of  coolant  accident  (LOCA),  simulation 
requires  quality  control  improvement .  The 
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use  of  process  control  checklists  has  been 
successful.     These  lists  are  orderly  checks 
of  test  set-up,  instrumentation  operation, 
test  sequences  and  ancillary  and  monitoring 
equipment  to  observe  and  record  data  and 
specimen  performance. 

3.1. 7.  It 

Contractor  responsibilities  include 
pre-test  quality  planning  for  the  same  rea- 
sons mentioned.     Quality  personnel  must  be 
involved  at  the  earliest  possible  time. 

3.1.7-5 

Test  deviations  in  laboratory  instru- 
mentation and  test  specimen  failures  re- 
quire immediate  verbal  notification  and 
laboratory  quality  procedures  should  re- 
quire written,  timely  submission  of  details, 
including  cause  and  corrective  action,  with 
provision  for  contractor  engineering  and 
quality  personnel  approval  signatures  as  ap- 
plicable . 

3.1.7.6 

In  testing  to  verify  design  of  safety 
related  devices  under  10CFR50,  reporting  of 
defects  and  non-conformances  as  required  by 
10CFR21  will  be  imposed  and  provides  severe 
penalties  for  failure  to  report.  Laboratory 
customer  quality  personnel  must  monitor  to 
assure  performance  to  requirements  by  labor- 
atory personnel,  instrumentation  and  test 
specimens . 

3.1.7-7 

Test  data  are  the  end  product  purchased 
by  the  contractor.     Data  must  be  complete, 
accurate  and  legible.     A  chronological  test 
log  must  be  maintained  by  laboratory  per- 
sonnel with  entries  of  time,  date  and  sig- 
natures . 

3.1.7.8 

There  must  be  a  pre-stated  mutual  agree- 
ment for  data  retention.     Some  data  are  re- 
quired to  be  held  for  years  and  contractual 
relief  to  the  laboratory  should  be  given  in 
writing.     Original  data  being  turned  over  to 
the  contractor  one  year  after  approval  of 
final  test  report  has  worked  well. 

3.1.7.9 

Retained  data  in  the  laboratory  must 
be  protected  and  considered  proprietary. 
Pass  or  fail  information  and  results  are  of 
value  to  contractor  competition. 


3.1.7-10 

Review  of  test  data  must  be  made  at 
pre-determined  intervals  consistent  with 
the  agreed  upon  test  plan . 

h.  Conclusion 

h.l 

Therefore,  inclusion  of  those  items 
presented  and  in  particular,  the  pre-test 
quality  planning  efforts  will  assure  that 
products  tested  will  conform  to  advertised 
performance  specifications  and/or  con- 
tractual obligations. 
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The  platinum  resistance  thermometer  (SPRT) ,  calibrated  in  accor- 
dance with  the  specifications  of  the  International  Practical  Temperature 
Scale  of  1968  (IPTS-68),  is  the  international  standard  for  temperature 
measurement  in  the  range  13.81  K  (-259.34°C)  to  903.89  K  (630.74°C)  to 
which  the  United  States  subscribes.    Such  SPRT's  are  used  to  calibrate 
other  SPRT's,  "industrial  resistance  thermometers",  or  other  types  of 
thermometers,  or  used  directly  where  highly  accurate  temperature  mea- 
surement is  desired.    There  are  many  laboratories  in  the  United  States 
that  calibrate  SPRT's  and  other  resistance  thermometers  at  various 
levels  of  accuracy.    A  measurement  assurance  program  (MAP)  on  the 
calibration  of  SPRT's  has  been  developed  at  the  National  Bureau  of 
Standards.    In  the  program,  three  pre-cal i brated  SPRT's  are  shipped  to 
a  participating  laboratory  for  calibration.    Upon  return,  the  SPRT's 
are  recalibrated  at  the  NBS  and  shipped  to  the  next  laboratory.  The 
two  NBS  calibrations  (before  and  after)  are  compared  with  those  of  the 
participant.    This  paper  describes  the  results  of  the  MAP  measurements 
on  the  SPRT's. 

Key  Words:    Calibration;  check  thermometers;  fixed  points;  International 
Practical  Temperature  Scale  of  1968;  measurement  assurance  program; 
platinum  resistance  thermometer;  reference  thermometer;  thermometer. 


1.  Introduction 

The  National  Bureau  of  Standards  (NBS) 
has  the  responsibility  to  establish,  main- 
tain, and  develop  the  standards  for  tempera- 
ture measurements  for  the  nation's  industrial 
and  scientific  communities  and,  in  coopera- 
tion with  other  national  laboratories,  to 
establish  international  uniformity  of 
temperature  scale.    The  dissemination  of 
these  standards  and  the  methodology  of 
temperature  measurements  is  achieved  through 
appropriate  publications,  consultations, 
calibration  services,  Measurement  Assurance 
Programs  (MAP),  and  temperature  measurement 
seminars1.    The  International  Practical 
Temperature  Scale  of  1968  (IPTS-68)  is  the 
present  basis  for  internationally  uniform 
values  of  temperature  [1,2]. 2    The  IPTS-68 


is  based  on  the  assigned  values  of  the 
temperatures  of  specified,  reproducible 
equilibrium  states  of  pure  substances 
(defining  fixed  points)  and  on  standard 
instruments  calibrated  at  these  temperatures. 
Between  the  fixed-point  temperatures, 
interpolation  equations  relate  the 


because  of  the  increased  interest  in 
thermometry  the  last  few  years,  these 
precision  thermometry  seminars  are  now 
being  held  twice  a  year  at  the  NBS,  in 
March  and  in  September. 

2Figures  in  brackets  indicate  the 
literature  references  at  the  end  of  this 
paper. 


indications  of  the  standard  instruments  to 
temperature  values  of  the  IPTS-68.  The 
platinum  resistance  thermometer  is  the 
specified  standard  instrument  from  13.81  K 
(-259.34°C)  to  903.89  K  (630.74°C),  the 
platinum-10%  rhodium  alloy  versus  platinum 
thermocouple  from  903.89  K  to  1337.58  K 
(1064. 43°C),  and  the  optical  pyrometer  above 
1337.58  K.    The  assigned  values  of  tempera- 
tures of  thirteen  equilibrium  states  that 
define  the  IPTS-68  are  given  in  table  1  (the 
states  and  values  with  asterisks).  For  the 
specified  interpolation  equations  of  the 
various  temperature  ranges,  see  reference 
[2]. 

As  part  of  the  study  of  properties  of 
substances  or  of  the  direct  investigation  of 
other  suitable  thermometric  reference  points, 
temperatures  of  various  equilibrium  states 
other  than  those  of  the  defining  fixed 
points  are  constantly  being  published. 
(Research  is  conducted  at  the  NBS  to  improve 
the  realization  of  the  defining  fixed  points, 
as  well  as  other  suitable  fixed  points.) 
Most  of  the  temperatures  of  the  equilibrium 
states  are  obtained  with  thermometers  that 
have  been  calibrated  in  terms  of  the  IPTS- 
68.    Recently  the  Consultative  Committee  for 
Thermometry  (a  subcommittee  of  the  Inter- 
national Committee  of  Weights  and  Measures) 
published  a  compilation  of  such  measurements 
for  secondary  thermometric  reference  points 
between  13  and  3700  K  [3],    Values  up  to  the 
gold  point  (1337.58  K)  from  the  compilation 
are  included  in  table  1.    The  vapor  pressure 
versus  temperature  relation  for  some  sub- 
stances given  in  the  compilation  are  exclu- 
ded from  table  1.    These  secondary  reference 
points  may  be  employed  to  check  the  calibra- 
tion of  thermometers  calibrated  in  terms  of 
the  IPTS-68  or  used  to  calibrate  such 
thermometers  or  other  thermometers  for  use 
over  a  limited  temperature  range.    [In  the 
calibration  of  thermometers  other  than  those 
specified  by  the  IPTS-68,  the  temperature 
intervals  of  calibration  must  be  small 
enough  so  that  the  temperature  unit  (i.e., 
the  size  of  the  degree)  of  the  thermometer 
scale  is  the  same  as  that  of  the  thermometer 
specified  by  the  IPTS-68.] 

This  paper  is  a  brief  report  of  the  MAP 
involving  the  calibration  of  platinum 
resistance  thermometers  in  accordance  with 
the  specification  of  the  IPTS-68.  (Hence- 
forth, a  platinum  resistance  thermometer 
that  meets  the  IPTS-68  specifications  for  a 
standard  will  be  referred  to  as  SPRT).  The 
range  of  calibration  of  the  SPRT  that  will 
be  discussed  is  from  the  oxygen  point  [i.e., 
90.188  K  (-182.962°C)j  to  903.89  K.    This  is 
the  range  where  most  of  the  temperature 
measurements  are  made  and  where  the  long- 
stem  SPRT  is  normally  used.    Also,  for 


convenience,  this  MAP  concerning  the  calibra 
tion  of  SPRT's  will  be  referred  to  as  SPRT- 
MAP.    Since  the  calibration  of  the  SPRT  can 
be  accidentally  changed  during  use,  proce- 
dures for  continued  testing  of  the  integrity 
of  the  measurement  process  (internal  MAP)  is 
described  as  part  of  the  SPRT-MAP. 

SPRT's  are  delicate  instruments.  They 
must  be  handled  in  a  manner  that  avoids 
tapping  or  bumping  against  table  tops  or 
other  objects.    They  should  not  be  vibrated. 
When  used  properly,  they  will  give  accuracie 
of  ±  1  mK  over  most  of  the  specified  tempera 
ture  range.    Such  SPRT's  are  used  to  calibra 
other  SPRT's,  industrial  type  platinum 
resistance  thermometers  (PRT),  or  other 
types  of  thermometers  (e.g.,  thermistors, 
mercury  thermometers,  quartz  thermometers, 
thermocouples,  etc.)  or  used  directly  where 
the  highest  accuracy  of  temperature  measure- 
ment is  required  (e.g.,  standard  cell 
baths,  length  standards,  electric  power, 
calorimetry,  etc.).    Hence,  the  SPRT  is  the 
standard  to  which  temperature  measurements 
over  a  broad  range  is  referred.    A  MAP  is 
needed  at  every  level  of  thermometry  accurac 
whether  the  accuracy  that  is  claimed  is  in 
the  calibration  laboratory  or  in  the  user 
laboratory  or  plant.    Although  this  paper 
deals  with  SPRT-MAP,  the  concept,  the 
repeated  check  measurements,  and  the  documen 
tation  of  the  MAP  are  applicable  to  other 
levels  of  thermometric  accuracy. 

To  disseminate  the  IPTS-68  to  the 
thermometry  community,  SPRT's  of  various 
laboratories  are  calibrated  at  the  NBS  in 
accordance  with  the  IPTS-68  at  the  specified 
fixed  points  (oxygen  point,  triple  point  of 
water,  tin  point,  and  zinc  point).  There 
are  a  number  of  standards  laboratories  in 
the  United  States  that  calibrate  SPRT's  at 
accuracy  levels  of  0.01  to  0.001  K  for  in- 
house  use  or  for  other  organizations.  Their 
SPRT  calibrations  are  traceable  to  the  NBS 
through  their  reference  SPRT's  that  have 
been  calibrated  at  the  NBS.    These  labora- 
tories have  one  or  more  of  the  specified 
fixed  points,  or  have  baths  that  are  con- 
trolled near  the  fixed-point  temperatures, 
for  calibration  of  the  SPRT's.    The  operatic 
of  the  fixed  points  is  checked  or  standardiz 
using  the  NBS  calibrated  reference  SPRT. 
Where  baths  are  used,  the  reference  SPRT  is 
used  to  calibrate  the  test  SPRT's  directly. 
These  reference  SPRT's  are  handled  extremely 
carefully,  e.g.,  some  are  "hand  carried" 
between  the  NBS  and  the  standards  labora- 
tories.     However,  in  spite  of  careful 
documentation  of  traceability  of  the  refer- 
ence SPRT  to  the  NBS,  the  calibration  may 
change  or  there  may  exist  unknown  weak  links 
in  the  measurement  process  of  the  laboratory 
itself.    The  SPRT-MAP  has  been  designed  to 
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Reference  points  with  asterisks  (*)  are  the  defining  fixed  points  of  the  IPTS-68. 

^Reference  points  listed  without  the  asterisks  (*)  were  taken  from  reference  [3].  Where 
more  than  one  value  was  given,  the  average  was  taken.    Values  only  up  to  the  gold  point 
1337.58  K  are  given  here;  for  values  above  this  temperature,  see  references  [2,3]. 

c 

Except  for  triple  points  and  where  pressures  are  indicated,  the  values  of  temperature 
are  for  equilibrium  states  at  a  pressure  of  1  standard  atmosphere  (101,325  Pa). 

TP  =  triple  point,  BP  =  boiling  point  (vanishingly  small  vapor  fraction), 

CP  =  condensation  point  (vanishingly  small  liquid  fraction),  FP  =  freezing  point, 

SP  =  sublimation  point,  e-h^  =  equilibrium  hydrogen,  n-h^  =  normal  hydrogen, 

tr  =  solid  phase  transition,  and  MP  =  melting  point. 

eThe  Ar  (TP)  may  be  used  as  an  alternative  to  the  02  (CP)  and  the  Sn  (FP)  may  be 
used  as  an  alternative  to  the  steam  point  to  obtain  the  IPTS-68  in  terms  of  the  SPRT. 
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compare  the  output,  i.e.,  a  standards  labora- 
tory's SPRT  calibrations,  with  those  of  the 
NBS. 

2.    IPTS-68  in  the  SPRT  Range 

The  SPRT  temperature  scale  of  the  IPTS- 
68  is  based  on  the  resistance  ratio  W(T) 
defined  by 


W(T)  =  R(T)/R(273.15K), 


(1) 


where  R(T)  is  the  SPRT  resistance  at  IPTS-68 
temperature  T  (kelvins)  and  R (273 . 1 5  K)  is 
the  resistance  at  273.15  K  (0°C).  [The 
Celsius  temperature  t(°C)  is  defined  by 
t  =  T  -  273. 15. ]3    The  SPRT  resistor  must  be 
"annealed  pure  platinum",  supported  in  a 
"strain-free"  manner,  and  have  a  value  of 
W(100°C)  not  less  than  1.39250.    The  resis- 
tor must  be  hermetically  sealed  inside  a 
protective  sheath  filled  with  dry  gas.  The 
interpolation  equations  are  expressed  in 
terms  of  W.    Hence,  to  determine  the  value 
of  temperature  on  the  SPRT  scale,  the 
resistance  of  the  SPRT  must  be  measured  at 
the  temperature  of  interest  and  at  0°C. 

2.1  Interpolation  Equations 

From  0°C  to  630.74°C  (903.89  K) ,  the 
values  of  IPTS-68  temperature  are  defined  by 


t  =  V  +  0.045(t7^)(Xtt 
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where  t'  is  defined  by 


W(t')  =  R(t')/R(0°C)  =  1  +  At"  + 
2 

Bt'  . 

Equation  (3)  is  equivalent  to 
V  -  l[W(t')-l]  +  6(^) 

hoo°c 

where 


3Henceforth,  temperatures  will  be 
expressed  interchangeably  in  °C  and  K, 
based  on  the  definition. 


(3) 


(4) 


A  +  B  X  100°C  (5) 


and 


B(100°C) 

A  +  B  X  100°C 


Equation  (2)  is  independent  of  the  SPRT;  it 
is  intended  to  adjust  the  SPRT  temperature 
scale,  given  by  equation  (3),  to  be  closer 
to  the  thermodynamic  temperature  scale.  The 
thermometer  constants  R(0°C),  A,  and  B  are 
determined  from  calibration  at  the  triple 
point  of  water,  the  steam  point  or  the  tin 
point,  and  the  zinc  point.    (Henceforth,  the 
triple  point  of  water  will  be  abbreviated 
TP).    The  constants  a  and  6  are  derived  from 
the  constants  A  and  B  according  to  equations 
(5)  and  (6),  respectively.    When  the  steam 
point  (100°C)  is  used  in  the  calibration, 
the  constant  a  can  be  obtained  directly  from 
equation  (4) ,  i.e., 


a  =  [W(100°C)-1]/100°C  =  [R(100°C)  - 
R(0°C)]/R(0°C)  X  100°C  (7) 


After  the  thermometer  constants  A  and  B  are 
obtained,  tables  of  W(T)  at  integral  values 
of  t  are  calculated  for  the  SPRT  from  0°C  to 
631 °C  using  equations  (2)  and  (3). 

Below  0°C,  the  W(T)  relation  of  the 
SPRT  is  given  by 


W(T)  =  W*(T)  +  AW(T) 


where  W*(T)  is  the  reference  function 
defined  by 


20 


(8) 


T  =    I    a.[UnW*(T)  + 
j=0  J 

3.28)/3.28]J  K 


(9) 


and  AW(T)  is  a  deviation  function  which  is  a 
polynomial  in  T.    The  coefficients  of 

equation  (9)  are  given  in  reference  [2]. 
The  reference  function  was  developed  from 
the  results  of  comparison  of  SPRT's  against 
gas  thermometers.    The  range  from  13.81  to 
273.15  K  (0°C)  is  divided  into  four  sub- 
ranges, each  with  its  specified  deviation 
function.    The  deviation  function  for  the 
subrange  90.188  K  (-182.962°C)  to  273.15  K, 
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the  range  included  in  the  SPRT-MAP,  is  given 
by 


AW(T)  =  b(T-273.15  K)  + 

e(T-273.15  K)3(T-373. 1 5  K)  (10) 


where  the  constants  b  and  e  are  determined 
from  calibration  at  the  TP  and  the  measured 
deviations  AW  at  the  steam  point  (100°C)  and 
at  the  oxygen  point  or  the  triple  point  of 
argon.    If  the  tin  point  instead  of  the 
steam  point  is  used,  W(100°C)  for  the  SPRT 
should  be  calculated  from  equations  (2)  and 
(3).    (Note  that  for  the  tin  point,  t  = 
231. 9681 °C  but  t1  =  231.9292°C).    After  the 
thermometer  constants  b  and  e  are  obtained, 
tables  of  W(t)  are  calculated  for  the  SPRT 
from  -183°C  to  0°C  using  equations  (10), 
(9),  and  (8). 

2.2    Fixed  Points 

Of  the  four  fixed  points  of  the  IPTS-68 
that  are  involved  in  the  SPRT-MAP,  the  TP  is 
the  most  important.    Although  R(0°C)  for  the 
resistance  ratio  W  may  be  obtained  from 
measurements  at  the  ice  point,  in  routine 
measurements  it  is  most  accurately  obtained 
from  the  resistance  measurements  at  the  TP, 
R(TP).    In  an  intercomparison  of  fifteen  TP 
cells  at  the  NBS,  the  averages  of  readings 
on  each  cell  were  all  within  a  range  of  ±0.1 
mK  [4].    Also,  in  another  test,  the  range  of 
readings  obtained  over  three  days  on  each  of 
eight  TP  cells  was  0.01  mK  [5].    In  terms  of 
the  resistance  value  at  0°C,  0.01  mK  corre- 
sponds to  4  X  10"8  R(0°C). 

The  freezing  points  of  tin  and  zinc  are 
realized  at  the  NBS  by  using  samples  of 
these  metals  that  are  nominally  99.9999 
percent  pure.    These  samples  are  issued  as 
NBS  Standard  Reference  Materials  SRM-740  and 
SRM-741 ,  respectively.    The  freezing  points 
of  cells  that  have  been  assembled  using 
these  metals  have  been  found  to  agree  within 
±0.1  mK  [6,7].    During  the  freezing  process 
the  furnaces  are  controlled  about  1  K  below 
the  freezing  point  to  give  a  complete  freeze 
duration  of  14  to  16  hours.    Normally,  the 
SPRT's  are  calibrated  during  the  first  50 
percent  of  the  freeze,  during  which  time  the 
temperature  change  is  not  more  than  0.2  mK. 

At  the  NBS,  the  SPRT's  are  calibrated 
at  the  oxygen  point  by  a  comparison  method 
in  terms  of  reference  standard  SPRT's,  using 
a  copper  block  apparatus  that  is  maintained 
during  the  measurements  as  nearly  isothermal 
as  possible  close  to  the  oxygen  point.  The 
results  of  repeated  measurements  over  a  two- 
year  period  on  "check  SPRT's"  show  the 


standard  deviation  of  the  oxygen-point 
calibration  to  correspond  to  ±0.16  mK.  (See 
the  section  on  check  SPRT's.)    This  oxygen- 
point  calibration  can  be  compared  to  the 
calibration  methods  used  by  many  standards 
laboratories,  where  the  SPRT's  are  calibrated 
in  baths  that  are  controlled  close  to  the 
fixed  points  in  terms  of  reference  SPRT's. 

3.    Internal  MAP 

In  addition  to  international  comparison 
of  temperature  scales  by  interchanging 
calibrated  SPRT's,  the  NBS  maintains  an 
internal  SPRT-MAP.    This  consists  of  a  bank 
of  fixed-point  cells  (TP,  tin  point,  and 
zinc  point),  which  have  been  tested  and 
found  to  yield  temperatures  that  agree 
within  ±0.1  mK  (see  section  2.2),  and  a  bank 
of  reference  standard  SPRT's  (for  the  oxygen 
point  and  for  temperatures  between  13  and  90 
K).    The  results  of  the  measurements  with 
these  cells  demonstrate  that  the  fixed 
points  used  at  the  NBS  are  highly  repro- 
ducible and,  since  materials  of  exception- 
ally high  purity  are  used,  the  temperatures 
obtained  are  close  to  those  of  pure  sub- 
stances.   As  part  of  this  internal  SPRT-MAP, 
measurements  are  obtained  with  check  SPRT's 
whenever  the  fixed-point  devices  are  used  in 
SPRT  calibration.    The  following  is  a  brief 
description  of  the  internal  SPRT-MAP  at  the 
NBS  (for  details,  see  references  [8,9,10]). 

3.1  Check  SPRT's 

In  a  calibration  laboratory  it  is 
extremely  important  that  the  laboratory's 
own  measurement  errors  be  small  enough  so 
that  the  results  of  the  users  of  the  cali- 
brated instruments  would  not  be  affected  by 
those  errors.    This  means  that  the  measure- 
ment process  of  the  calibration  laboratory 
must  always  be  under  control  and  must  pro- 
duce results  that  always  lie  within  these 
allowable  limits  of  measurement  error.  (At 
the  NBS,  procedures  are  constantly  being 
sought  to  improve  its  measurement  accuracy 
and  to  eliminate  possible  accidental  cali- 
bration errors.)    In  order  to  establish  the 
validity  of  a  single  calibration  on  a  new 
SPRT,  i.e.,  that  the  variability  of  the 
measurements  is  within  the  allowable  limits 
of  measurement  error  and  that  the  cali- 
bration process  is  under  control,  there  must 
be  redundant  measurements  of  a  control  or 
check  SPRT.    (A  check  SPRT  is  defined  herein 
as  an  SPRT  used  to  monitor  the  measurements 
at  a  temperature.)    Any  abrupt  shift  in  the 
calibration  of  the  check  SPRT  would  indicate 
that  there  may  be  problems  with  the  fixed- 
point  devices  or  with  the  measurement 
instrument,  possible  change  in  the 
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calibration  of  the  reference  SPRT  or  the 
bath  temperature  control  is  not  operating 
properly,  or  that  the  check  SPRT  was  acci- 
dentally "bumped."    The  long  record  of 
measurements  on  the  check  SPRT  gives  infor- 
mation on  the  limits  of  calibration  error  of 
any  new  SPRT. 

The  procedure  that  is  employed  at  the 
NBS  to  monitor  the  calibration  of  every 
batch  of  SPRT's  is  as  follows.  Different 
long-stem  type  check  SPRT's  are  assigned  to 
the  zinc-point,  tin-point,  and  oxygen-point 
calibration  measurements.    To  obtain  the 
corresponding  values  of  W  at  these  fixed 
points,  the  resistances  of  the  check  SPRT's 
are  also  measured  in  the  TP  cell  with  every 
measurement  obtained  at  the  zinc,  tin,  or 
oxygen  point.    In  the  cases  of  the  zinc  and 
tin  point  measurements,  the  freeze  is 
initiated  using  the  check  SPRT  and  the  first 
equilibrium  readings  are  obtained  on  the 
check  SPRT.    This  is  followed  by  calibration 
of  the  test  SPRT's  (usually  six).    After  all 
of  the  test  SPRT's  are  calibrated,  the  check 
SPRT  is  read  again  in  the  freezing-point 
cell.    The  second  reading  with  the  check 
SPRT  for  the  given  freeze  must  not  differ 
from  the  first  by  more  than  0.5  mK.  Usually 
the  difference  is  not  more  than  0.1  or  0.2 
mK  because  the  calibrations  are  made  during 
the  first  50  percent  of  the  freeze  (see 
section  2.2).    Also,  the  readings  should  be 
consistent  with  those  obtained  in  earlier 
freezes.    (For  those  laboratories  that 
employ  NBS  calibrated  SPRT's  to  standardize 
the  freezing  points  of  tin  and  zinc  before 
calibrating  the  test  SPRT's,  the  reference 
SPRT's  can  serve  as  their  check  SPRT's.) 
With  the  oxygen-point  calibration  apparatus, 
a  second  reference  standard  SPRT  is  cali- 
brated in  terms  of  the  working  reference 
standard  SPRT  in  the  same  manner  as  the  test 
SPRT's.    The  standard  deviations  of  the 
values  of  W  of  the  check  SPRT's  obtained 
over  a  two-year  period  were  ±0.28  mK,  ±0.30 
mK,  and  ±0.16  mK  for  the  zinc,  tin,  and 
oxygen  point  calibrations,  respectively. 

3.2    Secondary  Reference  Points 

Secondary  reference  points  have  many 
applications.    If  the  calibration  of  a 
reference  SPRT  is  uncertain,  rather  than  to 
be  recalibrated,  it  can  be  tested  at  one  of 
the  secondary  reference  points  in  the  range 
where  temperature  measurements  are  to  be 
made.    To  detect  any  possible  accidental 
calibration  errors  at  the  defining  fixed 
points  (e.g.,  transcription  error)  in 
routine  calibrations,  measurements  at  one  or 
more  of  the  secondary  reference  points  could 
be  included  as  part  of  the  calibration 
process.    Although  the  calibration  would  not 


be  in  accordance  with  the  IPTS-68  specif i 
cations,  SPRT's  may  be  calibrated  at  the 
secondary  reference  points  for  limited 
interpolation  use.    Differences  between  sue 
secondary  calibrations  and  the  IPTS-68 
calibration  should  be  expected,  depending 
upon  the  error  in  the  temperature  value  of 
the  fixed  point;  upon  variations  in  the 
realization  of  the  fixed  point,  e.g.,  sampl 
purity  or  technique;  and  upon  differences  i| 
the  SPRT's.    To  minimize  the  error  in 
realization  of  the  secondary  fixed  points, 
sealed  secondary  reference  cells  are  being 
developed  at  the  NBS  [11,12]. 

The  selection  of  a  secondary  referena 
point,  instead  of  a  defining  fixed  point  fi 
testing  the  calibration  of  a  SPRT,  is 
expected  to  be  based  on  its  greater  conve- 
nience and  its  temperature  being  in  the 
region  where  measurements  are  to  be  made. 
However,  the  TP  is  a  highly  reproducible  a 
easily  realized  defining  fixed  point; 
measurements  at  the  TP  should  detect  any 
large  changes  in  the  calibration  of  the 
SPRT.    (In  the  calibration  of  SPRT's  at  th| 
NBS  the  readings  are  obtained  in  the  fol- 
lowing order:    R(TP),  R(Zn),  R(TP),  R(Sn] 
R(TP),  R(02),  and  R(TP).    When  the  SPRT's 

are  to  be  used  only  above  -50°C,  the  last  ;J 
two  readings  are  omitted.    Over  a  two-year 
period  the  average  of  the  standard  devia- 
tions of  three  or  four  R(TP)  readings  of 
SPRT's  that  were  calibrated  was  about  ±0.1 
mK,  which  indicates  the  stability  that  coi 
be  expected  in  SPRT's  over  a  short  time 
interval  [10].) 
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It  is  essential  that  a  laboratory 
engaged  in  the  calibration  of  SPRT's  have 
least  two  reference  SPRT's  and  several  che i>; 
SPRT's  to  monitor  the  calibration  process 
Before  one  of  the  reference  SPRT's  is  seni 
to  the  NBS  for  calibration,  it  should  be 
calibrated  in  terms  of  the  second  referenn 
SPRT  (or  else  the  calibrations  of  the  SPR' 
should  be  compared.)    Upon  return  of  the 
first  reference  SPRT  from  the  NBS,  it 
should  be  calibrated  again  in  terms  of  th^ 
second  reference  SPRT.    If  the  SPRT's  werf 
handled  with  sufficient  care  during  use, 
two  calibrations,  before  and  after,  shoul 
agree  within  the  measurement  uncertaintie 
(In  a  series  of  tests  over  six  years,  whe 
SPRT's  were  exposed  intermittently  to  -18 
0,  100,  232,  420,  450,  and  480°C,  the 
calibrations  at  the  fixed  points  did  not 
change  by  more  than  1  or  2  mK.)    Also,  if 
the  second  reference  SPRT  had  previously 
been  calibrated  in  accordance  with  the  IPjl 
68  (or  at  the  NBS),  the  two  calibrations 
should  agree  with  the  new  NBS  calibration 


lk2 


ic: 


i  li 


ncfi: 


f:: 


Accidental  agreement  of  the  two  user-labora- 
tory calibrations  can  result  if  the  same 
error  is  made  in  both  calibrations,  e.g., 
the  fixed  point  may  be  0.01  K  colder  than 
the  assigned  value.    However,  the  comparison 
with  the  NBS  calibration  should  reveal  this 
error,  i.e.,  the  measured  value  of  W  and  the 
value  of  W  given  in  the  NBS  calibration  will 
be  different  for  the  "temperatures."  When 
the  second  reference  SPRT  is  sent  to  the  NBS 
for  calibration,  similar  comparison  calibra- 
tions should  follow.    Hence,  by  such  pro- 
cedures a  "do-it-yourself"  internal  MAP  can 
be  established.    Obviously,  more  calibrated 
reference  SPRT's  can  be  used  to  improve  the 
comparison  statistics  of  the  reference 
;SPRT*s.    In  the  calibration  of  test  SPRT's, 
check  SPRT's  must  be  employed  to  monitor  the 
calibration  process  each  time.    The  results 
jon  the  check  SPRT's  can  also  support  the 
stability  of  calibration  of  the  reference 
SPRT,  particularly  if  the  two  SPRT's  are  of 
Ithe  same  quality.    (Hence,  as  previously 
tated,  the  measurement  results  on  the  check 
PRT's  serve  to  monitor  the  complete  calibra- 
tion process. ) 

The  SPRT-MAP  service  provided  by  the 
NBS  requires  calibration  of  three  NBS  owned 
5PRT's.    The  participating  laboratory  is 
requested  to  calibrate  the  SPRT's  by  pro- 
cures that  are  normally  employed  in  the 
laboratory  for  the  calibration  of  SPRT's  or 
,i n  measurements  using  SPRT's.    (Some  labora- 
tories use  SPRT's  principally  to  calibrate 
thermometers  other  than  SPRT's  and  only 
-arely  calibrate  SPRT's.)    Work  sheets  that 
accompany  the  SPRT's  request  information  on 
temperature  standards  that  are  employed 
[i.e. ,  fixed  points  and  reference  SPRT's 
With  their  calibration  constants),  geometry 
fhjjind  other  data  on  the  fixed  point  devices 
;;:  ;ind  baths  that  are  important  in  temperature 
:r  measurements  (e.g.,  immersion  depth,  use  of 
:  iluminum  or  graphite  bushing,  temperature 
.^equalization  blocks  in  baths,  etc.),  the 
;: pertinent  measurement  data,  and  the  final 
alibration  tables  that  are  normally  fur- 
n' shed  to  "customers."    Immediately  upon 
f  Receipt  of  the  SPRT's,  and  just  prior  to 
shipment  of  the  SPRT's  back  to  the  NBS,  the 
./■«■  aboratory  must  determine  the  R(0°C)  of  the 
■PRT's  at  two  currents  (1  and  2  mA).  These 
otJoeasurements  in  conjunction  with  the  calibra- 
tion data  will  show  (1)  whether  the  SPRT's 
.;piere  harmed  during  shipment,  (2)  whether  the 
PRT's  were  carefully  handled  by  the  labora- 
'notrfory»  (3)  whether  the  measurement  procedure 
-f  the  laboratory  is  standard,  and  (4) 

hether  the  resistance  unit  is  close  to  our 
national  unit  of  resistance.    When  the 
ions  Vibration  is.  completed,  the  SPRT's  are 
eturned  to  the  NBS  for  recal ibration  and 
phipment  to  the  next  participating  labora- 
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tory.    The  SPRT-MAP  comparison  is  usually 
reported  back  to  the  participating  labora- 
tory within  three  to  six  weeks  after  receipt 
of  the  calibration  results,  depending  upon 
the  calibration  situation  at  the  NBS  and 
other  factors. 

The  laboratories  that  have  participated 
thus  far  in  the  SPRT-MAP  are  AVCO  (Systems 
Division),  Boeing  Aerospace  Company,  Duke 
Power  Company,  Leeds  and  Northrup  Company, 
Pratt  and  Whitney  Aircraft,  Rockwell  Inter- 
national (Autonetics ) ,  Rosemount,  Inc., 
Sandia  Laboratories,  U.S.  Air  Force  Aero- 
space Guidance  and  Metrology  Center,  U.S. 
Army  White  Sands  Missile  Range,  U.S.  Army 
Metrology  and  Calibration  Center,  U.S.  Navy 
Eastern  Standards  Laboratory,  and  U.S.  Navy 
Western  Standards  Laboratory.    Two  of  the 
laboratories  have  completed  their  second 
SPRT-MAP.    The  range  of  accuracies  that  was 
desired  by  the  laboratories  was  about  ±0.01 
to  ±0.001  K;  a  few  laboratories  felt  that 
accuracies  within  ±0.1  K  were  satisfactory 
in  some  of  their  applications. 

As  part  of  the  SPRT-MAP,  calibration 
tables  of  W(T)  at  integral  values  of  temper- 
atures were  received  from  the  participating 
laboratories.    The  values  at  -183,  100,  232, 
and  420°C  were  compared  with  those  obtained 
at  the  NBS.    The  values  at  these  temperatures 
are  sufficiently  close  to  the  fixed  points 
(oxygen,  steam,  tin,  and  zinc  points, 
respectively)  to  consider  them  equivalent  to 
those  obtained  at  the  assigned  temperatures 
of  the  fixed  points.    Figure  1  shows  the 
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Figure  1.    Standard  deviations  of 
calibrations  of  various  labora- 
tories from  the  National  Bureau  of 
Standards  calibration  of  three 
SPRT's  near  the  fixed  points. 


standard  deviations  of  each  of  the  various 
laboratories  from  the  NBS  calibrations  for 
the  three  SPRT's  at  the  above  temperatures. 
Most  of  the  standard  deviations  are  shown  to 
be  within  ±  5  mK,  but  there  are  some  devia- 
tions that  are  substantially  larger.  Very 
few  laboratories  made  measurements  at  the 
steam  point  (the  measurements  with  the  tin 
point  cell  being  easier);  however,  the 
calibration  values  at  the  steam  point  are 
compared  for  all  of  the  laboratories. 

Many  of  the  laboratories  have  been 
shown  to  be  performing  excellent  measure- 
ments; the  SPRT-MAP  results  have  added 
confidence  to  their  work.    Where  problems 
occurred,  such  as  adjustments  for  differ- 
ences in  barometric  pressure  on  the  freezing 
point  or  for  radiation  losses,  they  were 
readily  detected  and  corrected.    The  SPRT- 
MAP  work  sheets  helped  to  document  better 
the  measurement  process.    Where  the  SPRT-MAP 
showed  that  the  measurements  are  not  up  to 
expectation,  it  has  helped  to  indicate 
possible  improvements  in  the  measurement 
process. 

5.  Conclusion 

To  maintain  high  quality  measurement 
assurance  in  SPRT  calibration,  it  is  essen- 
tial to  have 

1.  )    temperature  standards  (fixed 

points  or  reference  SPRT's)  that 
are  compared  regularly  in  terms  of 
the  national  standards, 

2.  )    redundant  sets  of  measurements  to 

test  continuously  whether  the 
calibration  process  is  under 
control  (e.g.,  measurements  on 
check  SPRT's  that  are  always 
included  in  the  group  of  SPRT's 
being  calibrated), 

3.  )    comparison  of  the  calibration 

(i.e.,  the  output)  with  the  NBS. 

All  three  support  each  other  to  attain 
measurement  assurance. 

For  users  of  calibrated  SPRT's  (i.e., 
those  who  are  not  necessarily  involved  in 
the  calibration  of  SPRT's)  the  calibration 
of  the  SPRT  may  be  checked  by  employing  one 
or  more  of  the  many  fixed  points  listed  in 
table  1  in  the  temperature  range  of  interest. 
Such  measurements  also  provide  more  detailed 
measurement  assurance. 

6.    Future  Thermometry  MAP 

A  number  of  improvements  will  be  made 
in  the  current  SPRT-MAP  service  provided  by 
the  NBS,  particularly  in  the  area  of  pro- 


viding a  documentation  of  the  SPRT-MAP 
concept  and  measurement  procedure  to  serve 
as  a  guide  for  maintaining  a  good  SPRT-MAP. 
(For  details  of  the  measurement  process 
reference  should  still  be  made  to  [6,7,8, 
9,10].)    The  document  will  show  samples  of 
good  work  sheets  to  help  formulate  a  suit- 
able measurement  process.    The  document  will 
include  samples  of  internal  MAP  for  contin- 
uous monitoring  of  the  process  (see  section 
3.).    Most  of  the  laboratories  that  have 
participated  thus  far  in  the  SPRT-MAP  follow 
internal  MAP  procedures.    To  reach  a  broader 
thermometry  community,  an  industrial  platinur 
resistance  thermometer  MAP  is  being  developei 
to  serve  those  that  use  industrial  type 
PRT's  instead  of  SPRT's.    The  MAP  procedures 
are  expected  to  be  very  similar  to  those  of 
the  SPRT-MAP. 
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Based  on  a  survey  of  certification  for  building  products  United 
States,  HUD  has  developed  qualifications  and  procedures  for  admini- 
strators in  carrying  out  certification  programs.    In  addition,  a 
glossary  of  terms,  used  in  the  HUD  Certification  of  Building  Products 
Program,  has  been  developed.    In  the  United  States,  manufacturers 
provide  certification  of  a  product  to  a  particular  standard  while 
adminstrators  provide  a  validation  of  the  manufacturer's  certifica- 
tion.   Several  examples  of  certifications  will  be  discussed.    As  an 
adjunct  to  certification,  HUD  and  specific  program  administrators, 
have  developed  basic  criteria  to  be  used  by  administrators  in  approv- 
ing laboratories.    An  example  of  the  use  of  these  criteria  in  the 
carpet  program  will  be  extensively  discussed.    It  is  anticipated  that 
a  NVLAP  for  the  testing  of  carpet  will  be  developed.    Finally,  HUD's 
view  on  administrators,  building  products,  future  certification  and 
laboratory  accreditation  programs  in  the  United  States  and  inter- 
national certification  will  be  presented. 

Key  words:    Building  products;  certification;  laboratory  accreditation. 


Presently,  there  is  an  intense  interest 
in  the  certification  of  building  products  by 
code  officials,  architects,  and  government 
agencies  associated  with  the  building  indus- 
try.   The  purpose  of  this  paper  is  to  provide 
an  understanding  of  the  Department  of  Housing 
and  Urban  Development's  certification  of 
building  products  programs  and  to  provide 
some  basis  for  the  establishment  of  a  nation- 
al policy  in  the  United  States  for  certifica- 
tion of  building  products  and  for  other 
products  when  the  system  can  be  determined 
applicable. 

Certification,  of  a  product  to  a 
standard,  refers  to  the  authentication,  or 
attesting  by  a  manufacturer  or  vendor,  that 
the  product  being  sold  complies  with  a 
particular  standard.    It  is  important  to 
note  that  the  manufacturer  or  vendor  is  the 
only  one  that  can  make  this  declaration  or 
certification. 
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Figure  I  illustrates  three  types  of 
statements  of  compliance  found  in  certifi-' 
cation  programs.    The  first  type  is  the 
self-certification  statement  where  the 
manufacturer  declares  that  the  product  is 
in  compliance  with  a  standard.    For  exampl< 
the  manufacturer  makes  the  applicable  test: 
on  the  product  and  exercises  the  quality 
control  deemed  necessary  to  assure  that 
the  production  continues  to  comply  with  th  - 
standard. 
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The  second  type  is  similar  to  the 
first,  except  more  formal,  and  extensive 
records  are  kept  in  accordance  with  estab- 
lished pratices.  such  as  the  American 
National  Standards  Institute  Standard 
jZ  3^ .2  (which  requires  recording  of  the 
frequency  of  testing,  description  of  the 
manufacturing  process  and/or  quality  control 
programs. ) 

The  third  type  of  statement  of  compli- 
ance is  the  administrative  type  where  neutral 
third  parties  actually  collect ,  review  and 
file  test  reports;  check  the  manufacturer's 
quality  control  program  and  validate  the 
manufacturer's  certification. 

Third  party  organizations  or  administra- 
tors are  sometimes  used  to  conduct  certifi- 
cation programs  to  validate  the  manufacturer's 
declaration  of  certification.    Such  programs 
;  involve  required  test  frequencies  and  plant 
I  visits  to  assure  that  quality  control  proced- 
ures are  being  followed.    The  administrator/ 
/alidator  does  not  certify  products,  but 
-ather  gives  a  testimonial  based  on  test 
"ecords  and  inspections  that  the  manufacturer 
_s  complying  with  the  provisions  of  the 
standard.    The  administrator/validator  allows 
|;he  use  of  his  registered  mark  or  label  for  as 
!.ong  as  the  product  complies  with  the  standard 
ind  withdraws  the  use  of  the  mark  or  label 
lihen  noncompliance  is  established.    This  clear 
listinction  between  producer  and  the  admini- 
strator's involvement  is  important.  The 
■reducer  is  ultimately  responsible  for  the 
' ertif ication  of  the  product  and  the  admini- 
trator  is  responsible  for  the  conduct  or 
Operation  of  the  specified  certification  pro- 
gram.   Minimum  requirements  for  testing 
frequencies,  plant  visits,  etc.  are  set  forth 
n  the  certification  program  and  can  be 
ncreased  in  intensity  at  the  discretion  of 
•ne  administrator  relative  to  the  producers 
ability  to  comply  with  the  standard.  The 
ublic  is  informed  of  complying  products 
hrough  the  periodic  issuance  of  a  directory, 
Isting  or  marking  of  products.  Noncomplying 
roducts  are  deleted  from  such  issuances . 
pus,  it  is  only  the  producer  or  vendor  that 
ssumes  the  risk,  the  liability  and  the 
rofit  from  the  sale  of  the  product.  It 
lould  be  pointed  out,  however,  that  in  some 
oreign  countries,  the  government  itself  acts 
3  the  administrator  or  may  grant  a  waiver 
d  administrators  relieving  them  of  any 


liability  responsibilties.    In  these  cases 
the  administrator  can  also  certify  the  prod- 
uct along  with  the  producer. 

Appendix  I  contains  a  list  of  some  of 
the  third  party  organizations  validating 
building  products. 

Generally,  there  are  three  classes  of 
administrator  type  certification  programs. 
The  first  class  only  serves  as  the  manager 
or  administrator  of  the  program  but  does  not 
do  any  testing.    In  this  case,  the  admini- 
strator approves  other  laboratories  to  do 
the  testing  and  makes  audits  to  verify  the 
quality  control  level  in  the  plant.  The 
second  class  of  administrators  do  not 
approve  other  test  laboratories  but  does  all 
of  the  testing  itself.    Finally,  class  three 
administrators  are  characterized  by  having 
an  open  system  where  a  nonprofit  organiza- 
tion is  chartered  to  sponsor  a  program  and 
the  administration  is  put  out  for  bid.  An 
example  of  this  type  of  program  is  the 
Safety  Glazing  Certification  Council  (SGCC) 
program  and  the  Insulating  Glass  Certifica- 
tion.   Council  (IGCC)  program  for  insulating 
glass.    In  some  instances,  there  are  com- 
binations of  types  within  a  sample  certifi- 
cation program. 

Certification  programs,  involving 
administrators  or  third  party  type, 
generally  perform  the  following  ten  duties: 


1.  Administrates  a  program  relating  to  a 
specific  standard; 

2.  Obtains  a  license  agreement  between  the 
manufacturer  and  the  administrator; 

3.  Approves  laboratories  for  testing; 

4.  Reviews  initial  test  data  and  quality 
control  data  of  manufacturers  to  deter- 
mine acceptance; 

5.  Issues  notices  of  validation; 

6.  Allows  the  use  of  administrators'  labels 
on  manufacturers  products; 

7.  Issues  a  directory  or  listing  or  prod 
ucts  in  the  program  when  applicable; 


8.  Samples  and  periodically  visits  the 
plant  to  determine  continuing  compli- 
ance with  the  standard; 

9.  Decertifies  products  that  do  not  com- 
ply with  the  standard  after  intiial 
acceptance ; 

10.    Maintains  an  appeal  process  or  chal- 
lenge procedures  in  cases  of  dispute. 

In  order  to  clarify  many  of  the  other 
terms  used  in  certification,  a  list  of  terms 
used  in  the  HUD  Building  Products  Certifica- 
tion Programs  appears  in  Appendix  II. 

The  use  of  certification  programs  is 
growing  in  acceptance  because  they  are  meet- 
ing the  needs  of  the  vaious  segments  of  the 
building  industry.    One  of  the  main  purposes 
of  any  certification  program  is  to  provide 
some  assurance  to  the  user  that  the  product 
actually  delivered  complies  with  the  desig- 
nated standard  and  continues  to  do  so  on  an 
ongoing  basis. 

For  manufacturers,  certification  removes 
the  likelihood  of  unfair  competition  based  on 
spurious  claims  and  provides  for  a  fair  and 
equitable  basis  for  marketing  a  product. 
Certification  programs  also  assist  the  ac- 
ceptance and  establishment  of  new  products. 
Many  times,  new  products,  without  a  long 
history  of  performance,  are  not  accepted  by 
users.    Certification  can  provide  some  infor- 
mation and  assurance  that  the  product  at 
least  complies  with  a  standard  and  that  a 
quality  control  program  exists  for  continued 
production.    Often  this  assurance  can  be 
used  in  place  of  a  long  history  of  usage. 
Obviously,  however,  certification  programs 
can  never  guarantee  100  percent  assurance  or 
performance  but  can  provide  users  with  some 
degree  of  confidence  that  the  product  com- 
plies with  the  standard  and  will  perform 
satisfactorily  provided  that  the  reference 
standard  is  meaningful. 

Certification  programs  also  provide 
manufacturers  with  a  creditable  assurance 
for  their  products  that  they  alone  could  not 
provide.    These  programs  can,  in  addition, 
provide  some  assistance  to  manufacturers' 
quality  control  programs  and  prevent  con- 
fusion and  misrepresentation  to  consumers 


with  regard  to  claims  made  for  the  product. 
In  addition,  certification  programs  used  in 
conjunction  with  viable  standards  reduce  the 
chances  of  premature  failure  thereby  enhanc- 
ing cost  effectiveness  and  conserving  energy 
when  a  product  fails  and  has  to  be  replaced. 
Not  only  is  the  cost  of  the  produce  and  la- 
bor involved  but  also  the  loss  of  energy 
used  to  manufacture  the  product.  Consumers 
also  benefit  directly  because  it  aids  in 
making  value  judgment  choices  and  reducts 
the  need  for  duplicate  evaluations  by  dif- 
ferent users,  by  providing  impartial  and  ac- 
curate information  on  the  product.  Many 
times  these  judgment  choices  involve  concerns 
for  health  and  safety. 

A  classic  example  of  the  use  of  certi- 
fication program  in  achieving  many  of  the 
benefits  described  above,  is  the  HUD  Carpet 
Certification  Program.    Originally,  60  per- 
cent of  the  carpet  provided  to  HUD- insured 
mortgage  homes  did  not  comply  with  the 
existing  standard  even  though  manufacturer 
declared  their  carpets  complied  with  the 
standard.    After  one  year,  many  of  these 
carpets  deteriorated  and  necessitated  costly 
replacement.    Many  honest  and  reputable  car- 
pet manufacturers  lowered  their  quality 
levels  because  they  could  not  compete  on  an 
equitable  basis  with  manufacturers  that  did 
not  actually  comply  with  the  standard.  The 
purchase  of  carpet  was  particularly  diffi- 
cult for  users,  because  at  the  time  of  de- 
livery it  was  impossible  to  visually  deter-  ; 
mine  if  the  carpet  was  in  compliance  with 
the  required  standard.    With  the  advent  of 
a  carpet  certification  program,  the  per- 
centage of  carpet  in  the  HUD  program  not 
complying  was  reduced  to  less  than  seven 
percent.    Most  of  the  seven  percent  noncom- 
plying  carpets,  however,  involved  only  a 
few  factors  which  were  of  a  marginal  fail- 
ure nature,  consequently,  the  amount  of  car- 
pet that  would  constitute  any  major  failure 
would  be  negligible.    It  appears  this  level  j 
of  compliance  cannot  be  lowered  without  a 
more  intensive  testing  and  monitoring  pro-  , 
gram  which  would  be  extremely  expensive  for 
manufacturers  and  consumers.    We  do  not  be- 
lieve this  is  necessary. 
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Basically,  the  cost  of  the  carpet 
certification  program  to  the  manufacturer 
has  been  less  than  a  cent  per  square  yard  of 
carpet.    The  program  is  generally  accepted 
by  the  carpet  industry  with  about  80%  of  the 
mills  participating.    Many  skeptics  thought 
the  program  would  stifle  innovation  by 
imposing  new  demands  for  rigid  quality  con- 
trols but  each  year  the  number  of  qualities 
in  the  program  continues  to  grow.  Architects, 
designers  and  others  are  finding  the  benefits 
of  this  program  are  many  are  using  the  admini- 
strator directories  in  choosing  a  carpet. 

In  balance,  carpet  manufacturers  have 
been  pleased  with  the  results  of  the  program 
because  they  can  compete  on  a  fair  and 
equitable  basis.  Concomitantly,  the  growth  of 
carpet  sales  has  continued.    Consumers  are 
also  content  with  the  program  because  they  can 
determine  characteristics  of  any  carpet  in  the 
program  by  checking  the  marked  number  on  the 
back  with  the  listing  in  the  administrator's 
directory.    In  addition,  a  challenge  pro- 
cedure and  the  testing  of  field  samples  has 
added  creditability  and  prestige  to  the 
program. 

One  of  the  key  elements,  and  the  most 
expensive  part  for  an  administrator  to  carry 
out  in  a  certification  program,  is  approval  or 
accreditation  of  testing  laboratories. 
Generally,  the  following  criteria  are  used  for 
approving  or  accrediting  laboratories  in  con- 
junction with  a  proficiency  or  collaborative 
reference  program  and  an  initial  audit  visit. 

1.  Description  of  the  laboratory  including 
name  and  address  along  with  an  organiza- 
tional chart  of  the  laboratory; 

2.  Declaration  by  the  laboratory  of  its 
ability  and  services  to  be  offered; 

3.  Possession  of  a  laboratory  operations 
control  manual; 

4.  Submission  of  the  name  and  qualification 
of  responsible  personnel  including  a 
record  of  training  and  an  examination  of 
competency; 

5.  Description  and  photograph  of  equipment 
including  calibration,  verification  and 
maintenance  records; 


6.  Examination  of  the  laboratory  for 
environmental  and  safety  concerns; 

7.  Possession  of  sample  test  reports 
including  procedures  for  testing  and 
copies  of  all  relevant  standards; 

8.  Possession  of  a  record  of  an  appeals 
process  available  to  users  in  cases 
of  dispute. 

Our  programs  require  that  the  labor- 
atories be  approved  only  for  the  particular 
tests  they  are  able  to  conduct.  This 
follows  the  NVLAP  concept  which  the  Depart- 
ment of  HUD  has  encouraged  and  supported 
by  submitting  a  carpet  laboratory  approval 
program  for  the  Department  of  Commerce 
action. 

Internationally,  there  are  several 
groups  engaged  in  certification  activities. 
The  International  Standards  Organization 
(ISO)  has  established  a  technical  committee 
(Certico  TC  149)  to  address  the  concerns 
of  certifying  products  on  a  world  wide 
basis.    This  Committee  has  issued  several 
documents  including  guides  for  the 
principles  and  practices  of  certification 
along  with  guidelines  for  assessing  the 
technical  competency  of  testing  laboratories. 
One  of  the  goals  of  Certico  is  to  develop  a 
universal  system  for  the  acceptance  of 
products  complying  with  ISO  standards.  This 
system  would  remove  trade  barriers  and  reduce 
the  need  for  redundant  testing  by  several 
countries.    Presently  the  committee  is  work- 
in  four  areas:    (1)  Self  and  third  party 
certification;  (2)  Quality  assurance;  (3) 
Marking;  (4)  Basic  documents. 

It  is  anticipated  that  a  product  for 
international  certification  will  be  suggested 
in  the  coming  year.    Along  with  Certico,  an 
ad  hoc  committe  on  International  Laboratory 
accreditation  (ILAC)  is  developing  procedures 
for  accrediting  laboratories.    Their  efforts 
currently  are  confined  to  (1)  legal  problems 
of  accrediting  laboratories;  (2)  organizing 
a  directory  of  organizations  accrediting 
laboratories;  and  (3)  obtaining  a  list  of 
terms  and  needs  of  laboratory  accreditation. 
Finally,  there  is  an  effort  by  the  United 
Nations  (UN)  to  develop  harmonization  and 
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control  rules  for  buildings  and  building 
products.    While  efforts  of  these  organi- 
zations are  long  term,  it  appears  they 
are  progressing  at  a  rapid  rate.    It  is 
anticipated,  however,  that  the  immediate  form 
for  certifying  products  internationally  will 
be  bilateral  memoranda   of  agreement  letters. 

Looking  at  the  future  of  certification 
in  the  United  States,  it  appears  there  is  a 
need  for  a  national  policy  for  the  certifi- 
cation of  products.    One  method  to  initiate 
this  effort,  is  to  obtain  an  intergovern- 
mental agency  agreement  on  a  standard 
criteria  for  the  certification  of  products. 
It  is  very  frustrating  for  manufacturers  and 
consumers  to  determine  the  necessary 
requirements  for  a  particular  product.  For 
example,  insulating  materials  must  comply 
with  the  requirements  of  HUD,  DOC,  CPSC, 
DOD,  DOI,  etc.    What  is  needed  is  a  single 
document  that  contains  all  of  the  require- 
ments necessary  for  certification.    It  is 
HUD's  belief  that  the  responsibility  for 
building  products  rests  with  HUD.    This  does 
not  mean  that  the  functions  of  the  other 
agencies  are  usurped,  but  rather  there  should 
be  some  focal  point  on  certification  for 
manufacturers  of  building  products.  Equally, 
in  the  private  sector,  there  should  exist  the 
development  of  a  national  certification  com- 
mittee to  provide  for  a  unified  approach  on 
the  certification  of  products. 

We  believe  a  viable  single  system  for 
certification  and  laboratory  accreditation 
can  be  established  through  a  quasi  private 
sector-government  endeavor  that  would  be 
recognized  and  used  nationally.    Further,  it 
would  provide  a  base  for  international 
recognition  of  these  programs. 
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APPENDIX  I 


PRIVATE  BODIES  WITH  CERTIFICATION  PROGRAMS 


AAMA 

* 

Architectural  Aluminum  Manufacturers  Association 

ADL 

Associated  Dallas  Laboratories 

AGA 

American  Gas  Association 

AHAM 

Association  of  Home  Appliances 

APA 

American  Plywood  Association 

ARDL 

* 

Akron  Rubber  Development  Laboratory 

ARI 

Air-conditioning  and  Refrigeration  Institute 

ARL 

Applied  Research  Laboratory 

ASSE 

American  Society  of  Sanitary  Engineers 

AWIA 

# 

American  Wood  Inspection  Agency 

AWW 

* 

A.W.  Williams  Inspection 

AWWI 

# 

American  Wood  Window  Institute 

BHMA 

Builders  Hardware  Manufacturers  Association 

CLFMI 

Chain  Link  Fence  Manufacturers  Institute 

EPTL 

# 

El  Paso  Testing  Laboratory 

ETL 

* 

ETL  Testing  Laboratories 

HP  MA 

* 

Houston  Chemical  Service 

HVI 

Home  Ventilating  Institute 

IAPMA 

International  Association  of  Plumbing  &  Mechnical 

IGCC 

Insulating  Glass  Certification  Council 

ITL 

* 

Industrial  Testing  Laboratory 

JPMA 

Juvenile  Products  Manufacturers  Association 

MEA 

* 

Metallurgical  Engineers  of  Atlanta 

NAHB 

* 

National  Association  of  Home  Builders 

NPA 

* 

National  Particleboard  Association 

NSF 

National  Sanitation  Foundation 

NWMA 

* 

National  Woodwork  Manufactures  Association 

PFS 

Plywood  Fabricator  Service 

PTL 

Pittsburgh  Testing  Laboratory 

RIS 

Redwood  Inspection  Service 

SL 

* 

Skeist  Laboratories 

SPIB 

Southern  Pine  Inspection  Bureau 

SW 

Southwestern  Laboratory 

TCA 

Tile  Council  of  American 

TECO 

* 

Timber  Engineering  Company 

TPIT 

* 

Timber  Products  Inspection  &  Testing  Services 

UL 

Underwriters  Laboratories 

UST 

* 

U.S.  Testing 

WCLI 

* 

West  Coast  Lumber  Inspection  Bureau 

WGS 

* 

Williams  Grademarking  Services 

WHL 

* 

Warnock  Hersey 

WQA 

Water  Quality  Association 

WSC 

Water  Systems  Council 

WWPA 

* 

Western  Wood  Products  Association 

*  Registered  in  the  HUD  Building  Products  Certification  Program 
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APPENDIX  II 
HUD  BUILDING  PRODUCTS  CERTIFICATION  PROGRAM 
DEFINITION  OF  TERMS 


Acceptance  Sampling  Plan  -  A  specific  plan  that  states  the  sample  size  (number)  and  the 
associated  acceptance  and  rejection  criteria. 

Accreditation  Agency  -  An  organization  that  conducts  and  administers  an  accreditation  systeir 
and  recognized  to  have  the  competency  to  evaluate  that  testing  facilities  are  adequate  to 
conduct  tests. 

Accredited  Test  Facility  -  A  test  facility  which  has  been  approved  by  an  accreditation 
agency. 

Accreditation  -  A  public  declaration  by  a  national  agency  that  a  testing  laboratory  has 
the  qualifications  necessary  to  perform  a  specific  testing  activity. 

Administrator  -  Designated  by  the  sponsor  of  a  certification  program  to  perform  the  execute 
dut-tes    required  to  manage  the  affairs  of  that  program. 

Agency  -  An  organization  designated  by  some  authority  to  engage  in  testing,  inspection, 
and/or  administering  certification  programs. 

Applicant  -  The  person  or  (organization)  requesting  accreditation  of  its  certification 
program . 

Approval  -  The  acceptance  of  a  testing  laboratory  or  administrator  by  an  approving  body. 

Association  Test  Facility  -  A  test  facility  organized  and  operated  by  a  trade  or  profession; 
organization  and  providing  test  services  for  products. 

Audit  -  A  formal  verification. 

Authority  -  The  responsible  person  in  charge  of  the  work  or  the  authorized  representative.  [ 

( 

Calibration  -  The  comparison  of  two  objects,  one  of  which  is  of  a  known  or  accepted  values, 
to  determine  any  deviation  from  the  one  being  compared. 

Certification  of  Approval  -  A  seal,  statement  of  conformance,  label,  classifiction,  directo 
listing,  endorsement  or  other  affirmation  that  a  product  complies  with  applicable 
standards. 

Certification  -  The  procedures  by  which  products  or  services  become  certified. 

Certification  Mark  -  The  sponsor's  or  validator's  sign  or  symbol  that  identifies  a  product  I 
or  service  as  being  certified.    The  certification  of  compliance  is  by  the  producer  or  vendq 

Certification  Program  -  An  organized  system  where  products  or  services  may  be  certified  to 
specific  standards  on  a  uniform  equitable  basis. 

Certified  -  Attested  by  the  producer  or  vendor  under  the  procedures  of  a  certification 
program  as  satisfying  the  requirements  of  the  referenced  standard. 
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Certifier  -  The  producer  or  vendor  who  certifies  that  the  products  or  services  meet  the 
requirements  of  a  referenced  standard. 

Collaborative  Reference  Testing  -  A  program  where  a  uniform  material  or  a  products  of 
controlled  characteristics,  is  distributed  to  participating  laboratories.    The  laboratories 
test  the  materials  or  products  and  submit  the  test  results  for  evaluation. 

Commercial  Test  Facility  -  Provides  a  test  service  under  contract  or  on  a  fee  basis. 

Competence  -  Requisite  abilities  including  requirements  such  as  training,  experience, 
initiative,  supervision  needed  and  communication  skills. 

Compliance  Assurance  -  The  process  of  appraising  the  manufacturers  quality  control 
program  in  conjunction  with  monitoring,  auditing  and  inspecting  the  product  or  services 
being  offered. 

Conformity  Certification  -  The  action  of  certifying  by  means  of  a  certificate  or  mark  that 
a  product  or  service  conforms  to  a  standard. 

Consensus  -  Substantial  agreement  of  those  concerned  with  the  scope  and  provisions  of  a 
standard  as  judged  by  a  recongized  authority.    Consensus  implies  more  than  the  concept  of  a 
sample  majority,  but  not  necessarily  unanimity. 

Criteria  -  Detailed  requirements  for  examining  and  evaluating  a  laboratory  or  an  admini- 
strator to  assure  competency. 

Evaluation  -  A  formal  judgment  of  the  ability  of  a  testing  laboratory  or  administrator 
to  perform  satisfactorily  based  upon  analytical  data  or  other  judgment  factors. 

Evaulation  Team  -  A  group  of  individuals  recognized  as  competent  and  equipped  to  evaluate 
testing  facilities  or  act  as  a  referee  on  technical  questions. 

Field  Inspector  -  A  trained  person  capable  of  obtaining  samples  representative  of  production 
and  knowledgable  in  the  evaluation  of  quality  control  records,  test  methods  and  instrument 
calibration. 

Independent  Test  Laboratory  -  A  testing  laboratory  which  has  no    orgnization  tie  or  financial 
interest  in  the  manufacture,  vending  or  promotion  of  the  specific  product  or  services 
being  tested. 

Inspection  -  The  process  of  measuring,  examining,  gaging,  or  comparing  the  product 
or  service  with  the  specified  requirements. 

Inspection  Level  -  The  relative  sample  size  or  number  for  a  given  amount  of  production. 

Label  -  A  sticker,  nameplate,  or  similar  printed  statement  appearing  on  a  product  or 
packaging  material  which  provides  instructive  or  required  information. 

Laboratory  Accreditation  -  An  authoritive  action  for  evaluating  test  facilities  and 
approving  those  judged  competent  to  perform  specific  tests. 

Laboratory  Environment  -  The  aggregate  of  all  conditions,  such  as  temperature,  humidity, 
and  lighting,  that  may  influence  the  testing  or  performance  of  the  product. 
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Laboratory  Evaluation  -  A  process  of  inspecting,  examinining  or  assessing  the  organization 
to  determine  its  capability  and  competence  to  carry  out  specific  tests. 

Laboratory  Evaulation  Criteria  -  Statements  prescribing  the  organizational  resources,  the 
equipment  and  facilities,  the  operational  procedures,  and  the  minimum  techical  performance 
levels  required  of  a  testing  laboratory  for  accreditation. 

Laboratory  Examination  -  The  process  of  inspecting  and  obtaining  information  in  order  to 
judge  the  laboratory's  capability  for  performing  specific  tests. 

Licensee  -  The  manufacturer  or  vendor  authorized  by  contract  to  use  the  sponsor's  or 
administrator's  certification  mark  for  validation  purposes. 

Logo  -  A  symbol,  label,  hallmark  or  statement,  authorized  by  sponsor  or  administrator 
indicating  certification  or  compliance  with  the  specified  standard  by  the  producer  or  vendc 

Mandatory  Standard    -  One  which  is  established  by  law  and  the  observance  of  which  is 
compulsory. 

Manufacturer  -  See  Producer. 

Manufacturer's  Test  Facility  -  A  test  facility  organizationally  affilated  with  a  manufactur 
and  providing  test  services  on  the  product  and  services  of  that  manufacturer. 

Monitoring  -  The  act  of  observing  and  reviewing  certification  programs. 

Person  -  Any  individual,  consumer,  organization,  educational  intitution  corporation,  sociel 
Federal,  state  or  local  government  agency. 

Prerequisite  Testing  -  Preliminary  testing  carried  out  before  a  formal  collaborative  or 
other  testing  program  is  carried  out. 

Procedural  Guide  -  A  description  of  the  certification  program  containing  the  administratis 
testing  program,  standard  reference,  and  other  information  necessary  to  carry  out  a 
laboratory  or  administrator  function. 

Producer  -  A  person  or  orgnization  that  produces,  manufactures,  assemblies,  or  packages  a 
product  or  offers  a  service  that  is  capable  of  being  tested  for  conformance  with  the 
applicable  standard. 

I 

Product  -  Any  manufactured  item,  construction,  material  or  service  described  by  a  specific 
standard . 

Product  Certification  -  Attesting  product  conformance  to  a  referenced  standard  using  an 
organized  program. 

Proficiency  Testing  -  A  systematic  testing  program  where  a  sample  is  measured  by  a  number  : 
of  laboratories  or  a  random  series  of  samples  tested  to  determine  the  accuracy  of  measure- j 
ment. 

[ 

Qualified  Personnel  -  Competent  persons  adequately  trained  in  the  applicable  test  procedur 
equipment  operations,  and  calibration  methods. 

Quality  Assurance  -  A  systematic  approach  for  all  actions  necessary  for  the  manufacture  of 
a  product  or  performance  of  a  service  to  comply  with  a  designated  standard. 
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Quality  Control  A  means  by  which  a  manufacturer  achieves  conformity  to  a  given  standard 
to  prevent  the  production  of  defective  items. 

Reassessment  -  Repetition  of  the  complete  examination  and  evaluation  initially  required  of  a 
laboratory  or  an  administrator  seeking  accreditation. 

Redactory  or  Technical  Library    -  A  library  of  standards,  codes  and  other  technical  informa- 
tion necessary  for  carrying  out  a  testing  procedures,  and  verifying  that  the  product 
complies  with  standard. 

Reference  Facility  -  Recognized  as  professionally  competent     properly  equipped  and  staffed 
to  evaluate  the  technical  content  and  adequacy  of  a  test  method  and  acts  as  a  referee 
on  technical  questions. 

Reference  Material  -  A  material,  substance  or  device  whose  intrinsic  properties  are  used 
for  physical  comparison. 

Self-Certification  -  The  certifiction  of  a  product  by  a  producer  or  vendor  of  the  product 
being  certified. 

Sponsor  -  An  organization  under  whose  authority  a  certification  program  is  developed, 
promulgated,  and  financed  and  with  whose  name  the  certification  program  is  identified. 
It  may  also  own  or  be  in  process  or  registering  a  certification  mark. 

Standard    -  A  rule  for  course  of  action  established  by  an  authority,  custom,  or  general 
consent.    It  would  include  a  physical  embodiment  of  a  unit  of  measurement. 

Technician  -  An  employee  assigned  to  perform  the  actual  operations  of  a  testing  or 
isnpection  program. 

Testing  -  An  evluation  or  a  determination  of  a  physical  or  performance  characteristic 
for  the  purpose  of  establishing  conformance  to  a  standard. 

Testing  Agency  -  An  organization  staffed  and  equipped  for  measuring,  testing,  or  inspecting 
products . 

Test  Data  -  The  quantitative  and  qualitative  information  derived  from  testing. 

Test  Method  -  A  description  of  the  procedures,  equipment,  and  methodology  for  determining 
if  a  product  is  in  conformance  wholly  or  partially  to  a  standard  or  parts  of  a  standard. 

Testing  Facility  or  Laboratory  -  A  person  or  organization  whose  functions  include  testing, 
analyzing,  or  inspecting  products  or  services.    This  also  includes  the  physical  plant 
required  to  house  the  test  equipment  and  records. 

Third  Party  Certification  -  The  certification  of  a  product  by  an  organization  which  has  no 
organizational  tie  or  financial  interest  in  the  manufacture,  vending,  supplying  or  promotion 
of  the  product  being  certified. 

Traceable  to  a  Primary  Standard    A  documented  chain  of  comparisons  connecting  a  working 
standard  to  a  primary  standard  being  maintained  by  some  responsible  organization. 

User  -  An  organization  or  activity  which  makes  use  of  a  test  laboratory  or  an  accreditation 
agency . 
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LICENSING  PROGRAMS  FOR  FIELD  TECHNICIANS  AND  CONCRETE  LABORATORIES 

IN  MASSACHUSETTS 

GEORGE  H.  BRATTIN,  CHIEF  STAFF  ENGINEER 
CHARLES  G.  HANAFIN,  STAFF  ENGINEER 

THE  COMMONWEALTH  OF  MASSACHUSETTS 
STATE  BUILDING  CODE  COMMISSION 
BOSTON,  MASSACHUSETTS  02108 


The  state-wide  licensing  of  field  concrete  technicians  and 
concrete  testing  laboratories  has  been  in  effect  in  Massachusetts 
since  January,  1975.     As  of  this  date  there  are  twenty  licensed  con- 
crete testing  laboratories  and  approximately  1250  licensed  field 
concrete  technicians.     Prequalifying  agencies  were  designated  by  the 
State  Building  Code  Commission  to  insure  a  proper  mechanism  to  license 
technicians  and  laboratories.     Proven  test  methods,  procedures,  and 
standards  were  adopted,  wherever  possible,  to  give  the  programs  credence 
and  to  ease  their  implementation.     The  need,  development,  operation, 
rules  and  regulations,  and  evaluation  of  the  programs  are  set  forth. 

Key  words:  Accreditation;  concrete  testing  laboratories;  field  concrete 
licensing;  prequalifying  agency;  testing  agency. 


1.  Introduction 

The  first  known  state-wide  licensing 
programs  for  field  concrete  technicians  and 
concrete  testing  laboratories  were  imple- 
mented in  January,  1975,  by  the  promulgation 
of  the  Massachusetts  State  Building  Code 
under  the  jurisdiction  of  the  State  Building 
Code  Commission  as  authorized  by  the 
Massachusetts  legislation  (Chapter  802  of 
the  Acts  of  1972). 

Prior  to  this  time  there  were  no  re- 
quirements for  licensing  or  for  accredita- 
tion of  field  concrete  technicians  or  con- 
crete testing  laboratories  in  the  Common- 
wealth.    Also  there  were  no  uniform  building 
code  requirements  throughout  Massachusetts, 
which  proved  a  hardship  to  the  design  and 
construction  industry. 

Since  the  implementation  of  the  pro- 
grams, almost  five  years  ago,  twenty  con- 
crete testing  laboratories  and  over  1250 
field  concrete  technicians  have  been 
licensed  by  the  state. 
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2.     Need  for  Concrete  Accreditation 
Programs 

The  recommended  practices  of  the 
American  Concrete  Institute  and  other 
nationally  recognized  agencies  state  that 
inspection  agencies  should  be  hired  pri- 
marily on  their  capabilities  and  qualifi- 
cations, and  not  on  a  cost  basis.  Unfortu- 
nately, this  credo  often  gets  lost  in  the 
shuffle  and  inspection  contracts  often  go 
to  the  lowest  bidder,  who  may  or  may  not  be 
qualified,  unless  there  is  an  effective 
accreditation  or  licensing  program. 

The  final  acceptance  or  rejection  of 
concrete  in  a  structure  is  normally  based 
on  the  strength  of  the  concrete  meeting  or 
exceeding  a  specified  criteria  in  the  pro- 
ject specifications  which  is  based  on  the 
compressive  strength  of  cylindrical  concret 
specimens,  presumably  fabricated,  handled, 
cured,  tested,  etc.  in  accordance  with 
American  Society  for  Testing  and  Materials  I 
(ASTM)  standards,  some  twenty-eight  days 
after  the  concrete  is  incorporated  into  the 


structure.     If  the  procedures  of  the  ASTM 
standards  are  not  followed  (i.e.,  specimens 
are  improperly  fabricated,  handled,  cured, 
tested,  etc.),  the  cylinder  will  normally 
indicate  a  lower  strength  or  strength  po- 
tential than  concrete  properly  incorporated 
into  a  structure . 

Low  or  improper  strength  results,  in 
some  cases  caused  by  improper  inspection 
or  testing  procedures,  often  causes  delays 
in  construction,  load  tests,  lawsuits  and 
other  fallout  which  adds  to  the  cost  of 
construction  and  imposes  economic  hardships 
on  all  concerned  with  the  project. 

Other  concerns  over  the  reliability  of 
the  concrete  test  results  as  they  relate 
to  public  safety  and  structural  integrity 
are  simultaneously  brought  into  focus. 

Based  on  economics  and  safety  it  is, 
therefore,  important  and  essential  that  all 
field  concrete  technicians  and  concrete 
testing  laboratories  be  accredited  or 
licensed  to  ensure  that  they  are  qualified 
to  perform  the  required  tests  in  accordance 
with  the  applicable  standards  and  specifi- 
cations . 

3-     Implementation  Mechanism 

There  is  some  law  which  recognized  the 
impetuous  not  to  change  an  unsatisfactory 
condition  until  motivated  by  disaster.  In 
Massachusetts  this  motivating  disaster  was 
the  January,  1971  collapse  of  a  sixteen 
story  reinforced  concrete  apartment  struc- 
ture under  construction  at  2000  Commonwealth 
Avenue  in  Brighton  which  resulted  in  four 
fatalities.     The  resulting  press  coverage 
and  public  concern  lead  the  Massachusetts 
Legislative  to  propose  legislation  which 
would  prosecute  field  inspectors  for  not 
performing  all  the  tests  required  to  pre- 
vent such  an  occurrence  in  the  future. 

The  construction  industry  in  general, 
including  design  professional,  suppliers, 
and  testing  agencies  felt  that  the  proposed 
legislation  was  too  vague,  took  a  negative 
approach  to  the  situation,  and  would  have 
established  an  unneeded  commission,  which 
is  the  apparent  legislative  procedure  for 
dealing  with  a  crisis.     The  Massachusetts 
Construction  Industry  Board  (MCIB,  incor- 
porated as  The  Massacusetts  Concrete 
Industry  Board  before  broadening  its  scope) 
was  formed  to  speak  for  the  industry.  The 
State  Building  Code  Commission,  MCIB,  and 
other  concerned  agencies  and  individuals 
proved  successful  in  preventing  the  passage 
of  these  bills  based  on  their  commitment 


to  provide  positive  action  in  state-wide 
programs . 

The  field  concrete  technician  and  con- 
crete testing  laboratory  licensing  programs 
were  adopted  or  developed  by  the  Commission 
as  a  result,  based  on  the  developmental  work 
of  MCIB  and  assistance  of  the  National 
Bureau  of  Standards  (Contract  No.  5-35-776) 
[l]  Code. 

h .  Development 

The  provisions  of  the  licensing  pro- 
grams were  based  on  nationally  recognized 
standards,  practices,  and  procedures, 
wherever  possible,  to  provide  efficient  and 
reliable  programs,  free  of  specialized 
encumberances  and  related  specialized  costs. 

Key  to  the  programs  was  the  establish- 
ment of  a  prequalifying  agency,  recognition 
of  established  testing  agencies  to  examine 
and  evaluate  applicants,  and  the  adoption 
of  the  testing  agencies  proven  testing 
programs . 

k.l    Concrete  Testing  Laboratories 

The  Commission  designated  the  Cement 
and  Concrete  Reference  Laboratory  (CCRL)  as 
the  prime  testing  agency  to  examine  and 
evaluate  laboratories  desiring  to  be 
licensed. 

The  designation  of  the  nationally 
recognized  CCRL  lent  credence  to  the  pro- 
gram.    No  new  agency  or  new  testing  pro- 
cedures had  to  be  established.  Periodic 
reevaluation,  contained  in  the  program, 
was  made  to  conform  to  the  normal  CCRL 
inspection  tour  cycle. 

The  New  England  Division,  Corps  of 
Engineers,  also  has  been  designated  as  a 
testing  agency  to  perform  timely  examination 
and  evaluation  as  required  between  CCRL's 
normal  inspection  tours.     Due  to  the  nature 
of  their  work  and  reputation,  there  is  no 
conflict  of  interest.     Their  performance 
and  fees  are  compatible  with  CCRL. 

The  Construction  Materials  Safety 
Board  (CMSB)  was  established  by  the  Com- 
mission to  act  as  the  prequalifying  agency 
to  examine,  or  cause  to  be  examined,  the 
testing  agency  examination  and  evaluation, 
and  to  make  its  recommendation  to  the 
Commission  for  appropriate  action. 

CMSB  also  reviews  application  for 
materials,  devices,  products  and  methods  of 
construction  not  included  in  the  State 


Building  Code.     It  is  made  up  of  nine  mem- 
bers consisting  of  representatives  of  the 
Commission,  engineering,  architecture, 
testing  laboratories,  construction,  and 
university  faculty.     The  CMSB  performs  its 
services  voluntarily.     It  generally  meets 
one-half  day  per  month.     Administrative  and 
secretarial  duties  are  performed  by  Com- 
mission staff. 

The  Laboratory  Accreditation  Sub-Com- 
mittee was  formed  by  CMSB  to  act  on  its 
behalf  to  perform  the  detailed  evaluation 
of  the  testing  agency's  report  and  subse- 
quent examination  and  evaluation  of  the 
laboratory's  affidavits,  where  applicable, 
certifying  correction  of  deficiencies  cited 
in  the  testing  agency's  report.     In  some 
instances  the  Sub-Committee  requires  ad- 
ditional proof  of  deficiency  correction 
prior  to  making  its  recommendation  to  CMBS. 

The  Laboratory  Accreditation  Sub-Com- 
mittee is  made  up  of  five  members  consisting 
of  representatives  of  a  federal  laboratory, 
state  laboratory,  private  laboratory,  mate- 
rial supplier  and  Commission  Staff.  The 
makeup  of  this  Sub-Committee  provides  ex- 
pertise required  for  evaluation.     The  Sub- 
Committee  performs  its  services  voluntarily. 
It  generally  meets  one-half  day  per  month. 
Administrative  and  secretarial  duties  are 
performed  by  the  Commission  staff. 

k.2    Field  Concrete  Technicians 

The  Commission  designated  MCIB  as  the 
testing  agency  to  examine  and  evaluate  all 
persons  desiring  to  be  licensed  as  concrete 
field  technicians.     MCIB  submits  all  exami- 
nation results  and  evaluations  to  the  pre- 
qualifying  agency  for  their  action. 

MCIB  was  incorporated  in  1973  to 
foster  the  development  of  quality  construc- 
tion and  to  promote  and  support  high  stan- 
dards of  quality.     Its  membership  includes 
individuals,  firms  and  organizations  con- 
cerned with  construction,  design,  teaching, 
testing,  and  production. 

MCIB  developed  an  accreditation  program 
[2]  for  field  concrete  technicians  as  part 
of  its  commitment  to  public  safety  and  the 
legislature.     Previous  work  of  the  National 
Ready  Mixed  Concrete  Association  was  bene- 
ficial.    MCIB  began  its  examination  and 
accreditation  of  field  concrete  technicians 
in  the  summer  of  1973. 

MCIB  established  a  certification  com- 
mittee to  be  responsible  for  the  examination 
and  evaluation  and  its  reliability.     No  new 
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agency  had  to  be  established  and  the  Com- 
mission adopted  a  proven  working  program. 

CMSB  was  established  as  the  prequali- 
fying  agency  to  examine  the  evaluation  of 
the  testing  agency.     The  Commission  staff 
normally  acts  in  its  behalf  in  the  field  of 
concrete  technicians  programs. 

5.     Operation  of  the  programs 

3 

The  mechanism  for  the  operation  of  the 
licensing  programs  is  relatively  efficient. 

5.1    Concrete  Testing  Laboratory 

i 

1.     The  laboratory  makes  application  \ 
to  the  Commission  staff. 

2a.     The  laboratory  requests  testing 
agency  (CCRL  or  New  England  Division,  Corps  j 
of  Engineers)  examination  through  Commission 
staff  which  sometimes  directs  the  labora- 
tories to  the  testing  agency  directly. 

2b.     The  results  of  the  testing 
agencies  examination  are  forwarded  to  the 
prequalifying  agency  (the  Laboratory  Accred- 
itation Sub-Committee  of  CMSB)  through  the 
Commission  staff  within  ten  days  of  receipt 
by  the  laboratory. 

2c.     Any  laboratory  deficiencies  cited 
in  the  testing  agency's  report  are  to  be 
corrected  within  two  months  and  the  pre- 
qualifying agency  shall  be  so  notified  by 
affidavit  submitted  through  the  Commission  I 
staff. 

2d.     The  prequalifying  agency  normally: 
meets  within  the  month  to  approve  results 
and/or  corrections  to  any  deficiencies  so 
cited,  requests  for  more  information,  or 
recommends  denial  based  on  the  results  and 
lack  of  sufficient  information  received. 

3a.     The  laboratory  sends  the  prequaliH; 
fying  agency  a  notarized  statement  that  key 
positions  are  filled  by  qualified  personnel 
along  with  their  resume,  through  the  Com- 
mission staff.     The  key  positions  are  the 
director  of  testing  services,  supervising 
laboratory  technician,  and  supervisory  field 
technician.     One  person  may  fill  more  than 
one  position  if  qualified. 

3b.     The  prequalifying  agency  approves; 
the  personnel  criteria,  requests  more  infor-; 
mation,  or  recommends  denial  based  on  the 
information  submitted. 

h.     The  laboratory  sends  in  the  li- 
censing fee  (of  one  hundred  dollars  per 


year)  to  the  Commission  staff. 

5.  Upon  completion  of  the  prequali- 
fying  agency's  examination,  and  receipt  of 
the  licensing  fee,  action  is  taken  by  the 
Commission  on  the  issuance  of  the  concrete 
testing  laboratory  license. 

6.  The  licenses  require  renewal  an- 
nually.    The  laboratory  shall  send  the  pre- 
qualifying agency  (a)  an  affidavit  that  the 
compression  testing  machines  have  been  cali- 
brated and  verified  with  equipment  traceable 
to  the  National  Bureau  of  Standards.     (b)  A 
notarized  renewal  form  indicating  key  per- 
sonnel along  with  updated  resumes.     (c)  The 
licensing  fee. 

7.  The  prequalifying  agency  and  Com- 
mission take  appropriate  action  on  the  li- 
cense renewal. 

8.  The  laboratory  shall  notify  the 
Commission  within  seven  days  of  any  vacancy 
of  any  key  position  and  take  appropriate 
action  to  obtain  approval  of  a  person  to 
fill  that  position. 

5.2    Field  Concrete  Technician 

1.  The  person  requests  an  application 
form  from  the  Commission  staff. 

2.  The  person  makes  application  for 
examination  by  the  testing  agency  (MCIB), 
and  includes  the  examination  fee  (currently 
thirty- five  dollars). 

3.  The  MCIB  Certification  Committee 
notifies  the  applicant  of  the  next  field 
concrete  technician  examination.  (Special 
mini-examinations  can  be  held  in  case  of 
determined  hardship). 

h.  The  Certification  Committee  ar- 
ranges for  the  concrete,  place  of  tests, 
equipment,  and  jurors. 

5.     The  applicant  performs  physical 
tests  on  items  b  through  f  and  demonstrates 
knowledge  of  items  a  and  g. 

a.  ASTM  C1T2:     Sampling  Fresh  Concrete. 

b.  ASTM  Cli+3:     Test  for  Slump. 

c.  ASTM  C31:  Making  and  Curing  Test 
Specimens  in  the  Field. 

d.  ASTM  C231:     Test  for  Air  Content  - 
Pressure  Method. 


e.  ASTM  C1T3:     Test  for  Air  Content  - 
Volumetric  Method. 

f.  ASTM  C138:     Test  for  Weight  per 
Cubic  Foot  (Density). 

g.  ASTM  C192:     Storage  and  Transpor- 
tation of  Test  Cylinders. 

6a.     The  applicant  fails  the  exami- 
nation if  he  has  failed  two  of  the  perfor- 
mance tests,  b  through  f  above. 

6b.  The  applicant  is  given  a  "partial" 
if  he  fails  one  of  the  performance  tests  and 
must  take  a  retest  on  that  performance  test, 
in  order  to  pass. 

7.  Three  jurors  independently  grade 
the  applicant  as  to  pass  or  fail  on  each 
test.     The  jurors  are  accredited  field 
concrete  technicians  and  normally  represent 
three  different  disciplines  to  provide 
knowledgeable  and  unbiased  results.  The 
jury  system  is  the  backbone  for  the  reli- 
ability of  the  program. 

8.  At  least  two  members  of  the 
Certification  Committee  are  present  at  the 
examination  to  independently  evaluate  the 
applicant  as  to  passing  or  failing.  If 
the  Committee  is  not  in  concurrence  with 
the  pass  or  failure  of  the  candidate,  the 
jurors  are  consulted  which  normally  results 
in  resolution  of  the  situation.     If  the 
situation  cannot  be  resolved  it  is  for- 
warded to  the  Board  of  Trustees  of  MCIB 
for  their  action. 

9.  At  least  two  members  of  the  Board 
of  Trustees  of  MCIB,  being  elected  officials 
of  that  Board,  review  the  examination  re- 
sults for  final  evaluation  and  forward  the 
results  to  the  prequalifying  agency.  MCIB 
also  notified  the  applicant  of  MCIB  accred- 
itation or  failure. 

10.  The  accredited  applicant  sends  in 
a  picture  with  his  application  form  and 
licensing  fee  (currently  twenty  dollars  for 
two  years)  to  the  Commission  staff, 

11.  The  Commission  staff  normally  acts 
on  behalf  of  the  prequalifying  agency  (CMSB) 
and  forwards  the  results  to  the  Executive 
Director  of  the  Commission. 

12.  The  Executive  Director  issues  the 
licenses  for  those  who  qualify  upon  approval 
by  the  Commission. 

13.  Renewals  are  issued  bi-annually  by 
the  Executive  Director  upon  receipt  of  the 
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renewal  licensing  fee  (currently  twenty 
dollars ) . 

lit.     Concrete  Field  Technicians  work- 
ing for  government  agencies  are  exempted 
from  license  fees. 

6.     Rules  and  Regulations 

The  Commission  publishes  "Rules  and 
Regulations  for  the  Licensing  of  Concrete 
Testing  Laboratories"  [3]  and  "Rules  and 
Regulations  for  Concrete  Testing  Personnel" 
[It]  which  should  be  referenced  for  details. 
Many  of  the  requirements  contained  therein 
have  already  been  addressed.     Other  high- 
lights of  these  regulations  are  presented. 

6.1    Concrete  Testing  Laboratories 

1.     All  laboratories  including  branch 
and  project  laboratories  engaged  in  the 
testing  of  concrete  for  use  in  structures 
subject  to  construction  control  by  the 
State  Building  Code  shall  be  licensed. 
Structures  not  subject  to  control  by  the 
Third  (Latest)  edition  of  the  code  are: 

1.  Any  building  containing  less  than 
thirty-five  thousand  (355000)  cubic  feet  of 
space ; 

2.  Any  single  or  two  family  house 
or  any  accessory  building  thereto; 

3.  Any  building  used  for  farm  pur- 
poses; and 

It.     Retaining  walls  less  than  ten  (10) 
feet  in  height. 

2.  Except  as  modified  by  the  rules 
and  regulations,  all  concrete  testing  lab- 
oratories shall  conform  to  designated  pro- 
visions of  "Inspection  on  Testing  Agencies 
for  Concrete,  Steel  and  Bituminous  Materi- 
als as  Used  in  Construction,"  ASTM  E329-T2. 

The  personnel  qualification  require- 
ments are  broken  down  into  the  three  cate- 
gories established  by  ASTM  E329-T2,  that 
of  director  of  testing  services,  supervis- 
ing laboratory  technician,  and  supervising 
field  technician.     One  person  who  is  quali- 
fied may  fill  more  than  one  position.  The 
ASTM  E329-T2  criteria  have  been  broadened 
for  the  director  to  recognize  practical 
expertise  as  follows: 

The  testing  services  of  each  laboratory 
(main,  branch  or  project)  shall  be  under 
the  direction  of  a  Director  of  Testing 
Services  who  shall  be  a  full-time  resident 


employee  of  that  laboratory  and  shall  be 
qualified  in  accordance  with  any  one  (l)  of 
the  following  three  (3)  sets  of  require- 
ment s : 

a)  He  shall  be  a  Professional  Engi- 
neer, registered  in  the  Commonwealth  of 
Massachusetts  with  at  least  five  (5)  years 
of  experience  in  responsible  charges  of  worls 
related  to  Structural  Engineering,  Construc- 
tion Engineering  or  Construction  Materials 
Testing.     He  shall  be  subject  to  demonstrate 
his  ability  to  interpret  the  results  of 
tests  of  concrete  and  concrete  aggregates 
as  stated  in  ASTM  E329-T2;  or, 

b)  He  shall  have  a  Bachelor's  Degree  - 
in  Engineering  from  an  accredited  institu-  - 
tion  and  an  additional  total  of  three  (3) 
years  experience  performing  tests  on  con- 
crete and  concrete  materials  which  shall 
include  two  (2)  years  as  a  laboratory  tech- 
nician or  supervisor.     He  shall  be  subject 
to  demonstrate  his  ability  to  interpret  the; 
results  of  tests  of  concrete  and  concrete 
aggregates  as  stated  in  ASTM  E329-T2;  or, 

c)  He  shall  have  at  least  eight  (8) 
years  experience  including  five  (5)  years 
experience  as  a  laboratory  technician  or 
supervisor  and  shall  be  subject  to  demon- 
strate his  ability  to  interpret  the  results 
of  tests  of  concrete  and  concrete  aggre- 
gates as  stated  in  ASTM  E329-T2. 

It.     On  structures  subject  to  construc- 
tion control,  affidavits  must  be  submitted 
with  the  building  permit  application  to 
show  that  the  laboratory  is  licensed, 

5.  Provisions  are  provided  for  penal- 
ties and  revocation  of  licenses  for  false 
report  or  cause. 

6.  Any  aggrieved  laboratory  has  the 
right  to  appeal  to  the  State  Building  Code 
Appeals  Board  in  lieu  or  prior  to  any  court 
action . 

6.2    Field  Concrete  Technician 

1.  All  field  concrete  technicians 
engaged  in  the  testing  and/or  inspection 
of  concrete  in  structures  subject  to  con- 
struction control  (See  6.1  item  l)  shall 
be  licensed. 

2.  Provisions  are  made  for  revocation 
of  license  for  non-compliance  with  the  rule 
and  regulations,  Code,  or  standards  of  good 
practice . 
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3.     Any  aggrieved  person  has  the  right 
to  appeal  to  the  State  Building  Code  Appeals 
Board. 

7.     Evaluation  of  the  Field  Concrete 
Technician  and  Concrete  Testing  Laboratory- 
Programs  . 

This  evaluation  is  conducted  on  the 
eve  of  five  years  of  operation  of  the  field 
concrete  technician  and  concrete  testing 
laboratory  programs. 

7.1    Positive  Aspects 

1.  Parties  of  firms  involved  with  or 
licensed  by  the  program  believe  in  its 
merits  and  it  has  received  the  overall  sup- 
port of  the  industry. 

2.  The  operational  mechanism  and  rules 
and  regulations  have  had  only  minor  modifi- 
cations during  the  five-year  period.  This 
indicates  good  development  and  implementa- 
tion of  the  programs . 

3.  No  complaints  have  been  received  on 
the  caliber  and  performance  of  agencies 
(CCRL:  New  England  Division,  Corps  of  Engi- 
neers; or  MCIB)  or  the  prequalifying  agency 
(CMSB).     Their  results  are  evaluated  as  re- 
liable and  above  reproach. 

h.     Several  complaints  have  been  re- 
ceived by  MCIB  on  the  evaluation  of  the 
field  concrete  examination  by  applicants. 
All  except  one  were  withdrawn  upon  showing 
the  applicants  the  grading  on  the  jurors 
data  sheet.     In  the  one  exception,  the  Board 
of  Trustees  of  MCIB  voted  to  overrule  the 
findings  and  the  jurors  committee's  recom- 
mendations.    In  this  case  it  appears  that 
the  applicant  was  arrogant  and  may  have 
invited  harsh  critique  in  some  judgmental 
areas  by  the  jurors.     It  was  also  alleged 
that  one  juror  was  not  present  during  part 
of  one  test  and  relied  on  another  juror  for 
his  grading.     The  aggrieved  applicant  was 
able  to  convince  the  Board,  a  juror,  and  a 
Certification  Committee  member  of  his  thor- 
ough knowledge  and  ability  to  perform  the 
tests  during  this  hearing,  which  is  what  it 
is  all  about . 

5.  The  use  of  the  appeals  provision 
to  appear  before  the  State  Building  Code 
Board  of  Appeals  has  not  been  used  and  no 
litigation  has  been  entered  against  any 
portion  of  the  programs.     This  indicates  no 
major  grievances. 

6.  The  modifications  of  the  ASTM  E329- 
72  of  the  personnel  qualifications  for 


director  of  testing  services  has  allowed 
several  experienced  and  capable  persons  to 
continue  to  act  in  or  move  into  this  capac- 
ity.    This  would  not  have  been  possible 
under  that  standard  as  written. 

7.  A  better  appreciation  of  the  per- 
formance criteria  and  test  limitations  has 
been  realized.     Many  take  the  examination, 
pass  the  technician  examination  and  are 
accredited  by  MCIB  for  educational  or  other 
appreciative  reasons.     Many  accredited  tech- 
nicians live  outside  of  and  do  not  practice 
in  Massachusetts.     Many  accredited  techni- 
cians are  in  related  industry  and  not  test- 
ing. 

8.  Experienced  and  practicing  techni- 
cians had  a  failure  rate  of  one-third  on  the 
examinations  held  during  the  implementation 
of  the  program.     This  leads  to  the  supposi- 
tion that  at  least  one-third  of  the  field 
tests  were  performed  improperly  prior  to  the 
program  and  indicates  its  need  and  benefit. 

7.2    Areas  of  Concern 

1.  The  major  area  of  concern  has  al- 
ways been  that  of  monitoring  and  enforcing 
the  programs . 

Project  affidavits  are  required  to  be 
submitted  with  the  building  permit  applica- 
tion to  indicate  that  technicians  and  lab- 
oratories are  licensed  on  structures  subject 
to  construction  control.     This  places  the 
responsibility  of  providing  required  ser- 
vices on  the  person  who  signs  the  affidavit 
subject  to  persecution  if  the  terms  of  the 
affidavit  are  not  carried  out.     The  respon- 
sibility of  ensuring  that  the  project  affi- 
davit and  its  intent  are  carried  out  primar- 
ily rests  with  the  local  municipal  building 
official  and/or  the  state  inspector  who  re- 
views the  application  and  issues  a  building 
permit.     Building  officials  and  their  staff 
are  not  always  cognizant  of  all  details  of 
the  Code  let  alone  ancillary  rules  and  reg- 
ulations.    In  many  instances  they  do  not 
have  the  time  or  staff  to  effectively  moni- 
tor all  phases  of  all  the  construction  pro- 
jects in  their  jurisdiction.     One  municipal 
building  official  did  shut  down  a  project 
on  which  an  unlicensed  technician  was  per- 
forming as  a  licensed  field  concrete  tech- 
nician.    This  project  was  shut  down  for 
approximately  twenty  four  hours  until  a 
licensed  technician  was  present  at  the  site. 

The  program  further  relies  on  the  vol- 
untary self-policing  action  of  the  industry 
as  a  whole.     Suppliers  and  contractors 
should  want  to  see  qualified  technicians 


and  laboratories  on  the  project  so  as  not  to 
have  the  tests  indicate  lower  potential  of 
concrete  strength  than  that  incorporated  in- 
to the  structure.     Other  licensed  labora- 
tories and  technicians  should  want  to  see 
only  qualified  personnel  and  firms  active  to 
enhance  their  chances  at  obtaining  work  and 
also  enhance  the  professionalism  of  their 
field.     Sometimes  it  proves  difficult  to 
volunteer. 

Rumors  have  been  heard  of  violations: 

a.  Licensed  laboratories  have  sent  out 
unlicensed  technicians  to  projects  when  li- 
censed technicians  are  not  available,  in 
order  to  cope  with  the  variable  work  loads 
typical  to  the  testing  industry. 

b .  Field  equipment  has  been  used  that 
is  not  in  satisfactory  condition  to  pass 
CCRL  inspection,  and  which  may  not  provide 
reliable  results  within  designated  accu- 
racies, during  heavier  than  normal  work 
activity. 

c.  Licensed  technicians  do  not  always 
conform  to  the  recommended  testing  standard 
provisions  or  to  project  specifications. 
The  going  technician  pay  scale  (currently 
averaging  in  the  range  of  four  to  five  dol- 
lars per  hour)  may  not  always  invite  dedi- 
cated personnel. 

d.  The  morale  and  professionalism  of 
the  licensed  technicians  is  not  as  high  as 
it  was  during  the  early  stages  of  accredita- 
tion and  licensing. 

e.  One  laboratory  does  not  have  a  full- 
time  director  of  testing  services. 

Rumors  are  difficult  to  investigate. 
Formal  written  complaints  have  been  the  de- 
signated mechanism  to  initiate  investigation. 
Only  a  few  formal  written  complaints  have 
been  received  and  these  were  rated  invalid 
based  on  investigation  by  state  inspector  of 
the  Department  of  Public  Safety. 

More  attention  to  monitoring  and  en- 
forcement of  the  programs  by  local  and  state 
inspection  agencies  is  required  to  alleviate 
concern  over  rumors  and  to  take  action  on 
any  proven  violations. 

2.  The  staff  of  the  Commission  has 
been  honestly  criticized  in  not  taking  time- 
ly action  on  applications  and  other  matters, 
this  is  especially  true  during  times  of  code 
rewrite  and  specialized  projects.  This  is 
not  the  fault  of  the  program  but  one  of  work 
loads  and/or  administration.  Sufficient 


time  and  staffing  should  be  allotted  to  the 
program  in  order  to  readily  process  the  ad- 
ministration and  secretarial  work  on  a  week- 
ly basis . 

3.     No  training  program  is  part  of  the 
field  concrete  technician  licensing  program. 
Some  applicants  cannot  find  a  satisfactory 
way  to  prepare  for  the  performance  exami- 
nation.    Several  firms  provide  training  ser- 
vices (for  a  fee  on  the  order  of  fifty  dol- 
lars).    Some  do  a  good  job  but  others  do  not 
allow  enough  time  to  sufficiently  train  the 
applicant,  which  can  cause  an  economic  hard- 
ship (around  eighty  five  dollars)  on  the 
applicant  without  the  desired  results.  The 
MCIB  Certification  Committee  is  taking  this 
matter  under  advisement. 

h.     Not  all  of  the  Massachusetts  State 
Agency  Testing  laboratories  participate  in 
the  program.     The  proficient  agencies,  the 
Metropolitan  District  Commission,  the  Mass- 
achusetts Bay  Transportation  Authority,  and 
the  Mass  Port  Authority  do  participate. 
Notable  in  its  absence  is  the  Department  of 
Public  Works. 

5-     High  competition  for  work,  amongst 
the  laboratories,  reportedly  leads  to  econ- 
omic magic,  by  some,  in  order  to  break  even 
on  some  projects  or  to  survive.     The  pro- 
grams to  date  do  not  appear  to  have  fully 
alleviated  the  cost  shopping  practice. 

7.3    Cost  of  the  Program  on  the  Techni- 
cian and  Laboratory 

1.  The  field  concrete  technician  must 
pay  thirty  five  dollars  to  be  examined  and 
twenty  dollars  bi-annually  for  licensing. 
This  is  considered  reasonable. 

[ 

2.  The  only  cost  incurred  by  the  con- 
crete testing  laboratories  for  the  program 
is  the  one-hundred  dollar  annual  licensing 
fee.     The  one-thousand  dollars  for  the  bi- 
annual to  tri-annual  CCRL  inspection  and  the 
approximate  two-hundred  dollar  fee  for  annu- 
al compression  testing  machine  calibration 
are  considered  as  normal  operating  overhead 
for  proficient  laboratories. 

7.h    The  Cost  to  Administre  the  Program 

The  cost  to  the  Commonwealth  to  admin- 
ister the  program  is  minimal.     The  economi- 
cal administration  of  this  program  rests 
with  the  self-sufficiency  of  the  testing 
agencies  and  the  voluntary  services  of  the 
prequalifying  agency. 
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l.k.l    Concrete  Testing  Laboratory 

1.  CCRL,  the  designated  testing  agency, 
derives  its  own  funds  to  operate  its  func- 
tion. 

2.  The  Concrete  Sub-Committee  of  CMSB, 
the  prequalifying  agency,  volunteers  an  aver- 
age of  two  man-days  per  month  for  its  half 
day  monthly  meetings. 

3.  The  Commission  staff  averages  three 
to  four  man-days  per  month  to  administer  the 
program.     The  time  spent  is  higher  during 
CCRL  inspections  and  annual  renewals  hut 
tends  to  be  less  during  other  periods. 

f.k.2    Field  Concrete  Technician 

1.  MCIB,  the  designated  testing  agency, 
derives  its  own  funds  to  operate  its  func- 
tion by  the  thirty  five  dollar  examination 
fee,  which  was  established  on  a  break-even 
basis.     The  normal  cost  of  these  examina- 
tions is  in  the  order  of  two-thousand  dol- 
lars to  process  around  thirty  candidates. 
The  concrete  and  examination  facilities  are 
normally  donated.     The  biggest  expenditure 

is  for  a  fifty  dollar  honorarium  to  each 
juror.     Other  expenditures  include  coffee, 
lunch,  and  administration  costs. 

2.  The  Commission  staff  acting  for 
CMSB,  the  prequalifying  agency,  averages 
three  to  four  man-days  per  month  to  admin- 
ister the  program. 

7 . 5  Summary 

The  programs  have  provided  sense  of 
pride  and  professionalism  amont  the  techni- 
cians, better  appreciation  of  the  tests,  and 
more  reliable  results.     They  have  been  uni- 
versally accepted  within  Massachusetts,  and 
other  states  have  developed  programs  based 
on  them.     They  have  proven  efficient  and 
economical  to  administer. 


to  ensure  qualified  personnel  are  on  the 
project  to  oversee  proper  incorporation  of 
the  concrete  and  related  materials  into  the 
structure  to  realize  construction  and  speci- 
fied results.     Other  programs  under  consider- 
ation in  Massachusetts  are: 

Plant  Concrete  Technician 
Field  Concrete  Inspection 
Laboratory  Concrete  Technician 
Construction  Supervisors 

The  programs  are  currently  being  de- 
ferred in  favor  of  other  projects  and  bud- 
getary considerations.     It  is  hoped  that 
these  programs  can  be  developed  and  imple- 
mented without  the  need  for  a  disaster 
triggering  mechanism. 
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Uniform  criteria  has  been  established 
throughout  the  State.     The  programs  are 
worthwhile  and  have  been  proven  successful, 
but  require  more  vigilant  monitoring  and  en- 
forcement to  sustain  or  improve  upon  their 
rated  merits.     Other  planned  programs  are 
just  as  essential. 

8.     Other  Related  Programs 

The  implementation  of  the  field  con- 
crete technician  and  concrete  testing  labor- 
atory programs  are  a  couple  of  positive 
steps  in  ensuring  quality  concrete  construc- 
tion.    Other  programs  should  be  considered 
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THE  CRIME  LABORATORY  ACCREDITATION  PROGRAM  OF 
THE  AMERICAN  SOCIETY  OF  CRIME  LABORATORY  DIRECTORS  (ASCLD) 

ANTHONY  LONGHETTI 

DIRECTOR,  CRIMINALISTICS  LABORATORY 
SAN  BERNARDINO  COUNTY  SHERIFF'S  DEPARTMENT 
SAN  BERNARDINO,  CA  92415 

and 

CHAIRMAN,  ASCLD  COMMITTEE  ON 
LABORATORY  STANDARDS  AND  EVALUATION 


The  American  Society  of  Crime  Laboratory  Directors  (ASCLD)  has 
formulated  a  crime  laboratory  accreditation  program  with  the  following 
objectives:     1)  to  improve  the  quality  of  laboratory  services  provided 
to  the  criminal  justice  system,  2)  to  offer  to  the  general  public  and 
to  users  of  laboratory  services  a  means  of  identifying  throughout  the 
nation  those  laboratory  facilities  which  satisfy  accreditation  criteria, 

3)  to  develop  and  maintain  criteria  which  can  be  used  by  a  laboratory 
to  assess  its  level  of  performance  and  strengthen  its  operation,  and 

4)  to  provide  an  independent,  impartial  and  objective  system  by  which 
laboratory  facilities  can  benefit  from  a  total  organizational  review. 

A  significant  part  of  the  program  is  the  Standards  Manual,  which 
identifies  the  principles  and  standards,  discusses  them  and  lists 
evaluation  criteria  for  each  of  the  following  areas:     1)  laboratory 
management  and  operation,  2)  personnel  qualifications,  3)  procedures 
and  instruments /equipment,  and  4)  physical  plant  and  security. 

A  pilot  program  is  currently  under  way  to  test  the  on-site 
evaluation  aspect  of  the  accreditation  program.     Four  crime 
laboratories  of  varying  size,  geographic  location,  and  organizational 
placement  have  volunteered  to  serve  as  "test  sites"  for  this  phase  of 
the  program 

Key  words:    Accreditation;  American  Society  of  Crime  Laboratory 
Directors  (ASCLD);  certification;  management;  on-site  visit; 
personnel  qualifications;  physical  evidence;  proficiency  testing; 
standards. 
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1.  Introduction 


1.1  Definition 


The  National  Advisory  Commission  on 
Criminal  Justice  Standards  and  Goals  made 
the  following  recommendation  in  1973  in  the 
volume  titled  POLICE  [l]1: 

"(Recommendation  12.1,  Certification  of 
Crime  Laboratories) 

"It  is  recommended  that  a  national 
program  be  established  to  insure  that  all 
tests  and  analyses  performed  by  state, 
regional,  or  local  laboratory  facilities  are 
procedurally  sound  and  scientifically  valid. 
The  program  should  provide  for  the  certifi- 
cation (sic)  of  those  facilities  whose 
testing  procedures  and  scientific  analyses 
meet  the  minimum  standards  set  by  the 
agency  administering  the  program. 

"1.    An  existing  national  agency  or 
organization  should  be  designated  to 
administer  the  program.    This  body  should 
develop  minimum  standards  by  which  it  can 
measure  every  crime  laboratory's  level  of 
proficiency. 

"2.    The  national  agency  or  organi- 
zation should  conduct  periodic  evaluations 
of  every  state,  regional,  and  local 
laboratory  to  determine  its  level  of 
proficiency  in  performing  laboratory  tests. 
In  conducting  the  evaluation,  it  should 
rate  the  laboratory  only  on  the  basis  of 
those  tests  which  it  actually  performs  in 
rendering  services. 

"3.    The  national  agency  or  organi- 
zation should,  on  the  basis  of  the 
evaluation,  certify  every  laboratory  that 
meets  or  exceeds  the  designated  minimum 
standards  in  all  the  tests  which  it  per- 
forms. " 

The  results  of  a  LEAA-funded 
proficiency  testing  research  program  added 
objective  substantiation  to  the  intuitive 
feeling  of  many  practitioners  that  some 
forensic  science  laboratories,  for  varying 
reasons,  were  not  performing  all  physical 
evidence  examinations  at  a  level  consistent 
with  the  state-of-the-art  in  the  criminal- 
istics profession  [2],    One  general  finding 
of  the  report  was  that  "a  wide  range  of 
proficiency  levels  among  the  participating 
laboratories  exists,  and  in  general,  there 
are  several  evidence  types  with  which  the 
laboratories  are  having  serious  diffi- 
culties." 


^■Figures  in  brackets  indicate  the 
literature  references  at  the  end  of  the 
paper. 


The  National  Advisory  Commission  on 
Criminal  Justice  Standards  and  Goals,  as 
previously  noted,  used  the  term  "certi- 
fication" in  reference  to  the  process  by 
which  a  facility,  in  this  instance  a  crime 
laboratory,  is  evaluated  to  determine 
whether  or  not  it  meets  minimum  standards 
of  performance.    Clearly,  the  proper  term 
is  "accreditation",  defined  as:  "....a 
procedure  by  which  a  non-government  agency 
determines  that  a  program  of  study  or  an 
institution  meets  prescribed  standards.  The 
institution  may,  for  example,  be  a  college, 
a  university,  or  a  hospital."  £3] 

2.    The  ASCLD  Crime  Laboratory 
Accreditation  Program 

The  American  Society  of  Crime  Labora- 
tory Directors  (ASCLD)  was  organized  in 
1973.    One  of  its  principal  objectives  is  to 
promote,  encourage  and  maintain  the  highest 
standards  of  practice  in  the  field  of  crime 
laboratory  services.    The  Committee  on 
Laboratory  Standards  and  Evaluation  has 
progressed  toward  that  goal  by  developing  a 
Standards  Manual  and  by  working  to  establish 
an  accreditation  program  that  will  be 
financially  self-supporting  and,  consistent 
with  the  definition  of  accreditation,  be 
peer  administered. 

The  ASCLD  accreditation  program  is 
totally  neither  product  oriented  nor 
discipline  oriented.    The  crime  laboratory's 
"products"  that  are  tested  (specifically, 
they  are  identified,  sometimes  quantitated 
and  often  compared)  include  an  almost 
infinite  variety  of  items  that  potentially 
may  become  physical  evidence  in  a  criminal, 
or  civil,  investigation  and  trial.  Most 
commonly,  the  items  tested  include  firearms, 
documents,  bloodstains,  seminal  stains, 
drugs,  poisons,  hairs,  fibers,  paint,  glass, 
flammables,  explosives,  and  soils.  Less 
common,  but  by  no  means  unusual,  are  metals, 
vegetation,  paper,  wood,  plastics  and  saliva. 

The  term  "discipline",  as  applied  to  a 
crime  laboratory,  is  not  clearly  defined. 
Within  the  criminalistics  profession,  it  is 
consistently  applied  to  sub-specialties 
that  are  actually  groupings  of  physical 
evidence  types.    For  example,  personnel 
certification  efforts  now  in  progress  have 
defined  the  four  sub-specialties,  or 
"disciplines",  of  firearms  and  toolmark 
examination,  serology,  drug  (controlled 
substance)  identification,  and  "trace 
evidence"  (hair,  fiber,  glass,  paint, 
soil,  etc.) 


ASCLD  is  using  neither  the  relatively 
narrow  product -focused  approach  that  deter- 
mines only  a  crime  laboratory's  capability 
to  test  a  specific  item  of  physical  evidence 
(e.g.,  a  bloodstain)  to  some  specified 
standard  using  a  definitive  test  method,  nor 
the  equally  restrictive  discipline-focused 
approach  used  by  the  National  Association  of 
Testing  Laboratories  in  Australia  (such  as 
chemical  testing  or  instrumental  analysis). 
Rather,  it  has  selected  a  broad  accreditation 
program  that  identifies  principles  and 
standards  for  each  of  the  following  four 
areas  of  a  crime  laboratory's  operation: 

1)  laboratory  management  and  operations, 

2)  personnel  qualifications,  3)  procedures 
and  instruments /equipment ,  and  4)  physical 
plant  and  security. 

2.1    Laboratory  Management  and  Operations 

The  Laboratory  Management  and  Operations 
section  of  the  Standards  Manual  covers  the 
managerial  functions  of  planning,  organizing, 
directing  and  controlling.     The  planning 
function,  for  example,  is  defined  as  the 
analysis  of  relevant  information  from  the 
present  and  the  past  and  an  assessment  of 
probable  future  developments  so  that  a  course 
of  action  (plan)  may  be  determined  that 
enables  the  organization  to  meet  its  stated 
objectives.    The  objectives  are  then  outlined 
in  terms  of  principles  and  standards,  foll- 
owed by  a  brief  discussion  and  by  the  listing 
of  evaluation  criteria.    The  latter  are 
intended  for  use  by  the  laboratory  for  self- 
assessment  and/or  by  the  site  evaluation 
team  during  its  visit.    The  criteria  are 
1)  does  the  laboratory  have  a  stated  list  of 
objectives?  2)  have  the  objectives  been 
communicated  to  all  employees?  and  3)  do  the 
objectives  appear  relevant  to  the  needs  of 
the  community  serviced  by  the  laboratory? 

The  functions  of  organizing,  directing 
and  controlling  are  handled  in  the  same 
manner;  i.e.,  with  a  statement  of  the 
principle(s)  involved,  an  enumeration  of  the 
standards  as  they  apply  to  the  particular 
function  or  sub-function,  a  brief  discussion 
of  each  standard,  and  a  listing  of  the 
evaluation  criteria  for  each  function  or 
sub -function. 


2.2    Personnel  Qualifications 

The  qualifications  of  both  the  pro- 
fessional, and,  to  a  limited  extent,  the 
support  personnel  within  a  crime  laboratory, 
are  considered  and  judged  "in  context"  with 
the  situation  in  which  the  individual  works. 
Most  important,  therefore,  is  the  existence 
in  the  laboratory  of  a  system  such  that 
"reliability  is  assured  with  respect  to 
procedures,  equipment,  processing  and 
interpretation  of  results  and  that  the 
examinations  are  as  complete  as  adequate 
laboratory  equipment  permits  and  the  case 
situation  warrants."    If,  for  example,  in  a 
particular  type  of  examination  there  is 
close  supervision  by  a  qualified  forensic 
scientist,  the  need  for  a  highly  qualified 
person  doing  the  work  is  reduced,  assuming 
that  court  requirements  are  satisfied. 

Qualifications  are  considered  in  the 
ASCLD  program  for  personnel  in  the  following 
categories:     1)  management,  2)  controlled 
substances  identification,  3)  toxicology, 
4)  trace  evidence  examinations,  5)  serology, 
6)  firearms/toolmark  examination,  7)  docu- 
ment examination,  8)  latent  fingerprint 
examination,  and  9)  non-testifying  support 
personnel.    Using  serology  to  illustrate  th( 
principles  and  standards  as  stated,  the 
former  require  that  the  forensic  serologist 
should  have  knowledge  of  basic  biological 
sciences  sufficient  to  understand  the  basis 
of  forensic  serological  tests  and  sufficienl 
knowledge  of  chemistry  to  prepare  test 
specimens  and  solutions  and  to  understand 
the  mechanisms  involved  in  the  testing 
procedures.     In  addition  it  is  recommended 
that  the  forensic  serologist  have  adequate  , 
knowledge  of  the  statistics  (population 
frequencies  of  blood  groups)  in  the 
discipline  and  have  sufficient  mastery  of 
the  techniques  and  procedures  so  as  to  be 
able  to  produce  reliable  results.  The 
standards  are  stated  as  follows:     "1)  The 
forensic  serologist  should  have  completed 
the  science  requirements  for  a  baccalaureati 
degree  in  a  natural  science  or  in 
criminalistics,    2)    The  forensic  serologisi 
should  have  education  and  experience/trainii 
commensurate  with  the  analyses  and  testimony 
provided,    3)    Those  persons  conducting 
serological  examinations  should  have 
successfully  completed  an  extensive  series  . 
of  proficiency  tests,  and  4)    It  is 
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recommended  that  those  persons  conducting 
serological  investigations  meet  the  quali- 
fications of  an  appropriate  peer  group 
certification  board,  if  and  when  such  a 
board  is  formulated.     (Crime  Laboratory 
Accreditation  Program  -  rev.  5-15-79.) 

Standards  No.  3  and  No.  4,  in  regard  to 
proficiency  testing  and  certification,  are 
restated  for  every  one  of  the  personnel 
categories.    Because  certification  is  a 
voluntary  effort,  however,  there  is  a  state- 
ment in  the  introductory  part  of  the 
personnel  qualifications  section  to  the 
effect  that  absence  of  certification  of  an 
individual  does  not  necessarily  imply  that 
the  person  is  unqualified. 

2.3    Procedures  and  Instruments /Equipment 

The  thrust  of  this  aspect  of  the  ASCLD 
program  is  stated  simply.    Are  the 
procedures  and  instruments /equipment  used  by 
the  laboratory  either  generally  accepted  in 
the  field,  or  supported  by  adequate  scienti- 
fic data,  and  are  materials  necessary  to 
conduct  these  analyses  and/or  examinations 
available?    Thus,  the  criteria  for 
evaluation  require  that  a  crime  laboratory 
use  scientifically  acceptable  procedures  and 
prescribed  controls  and  standards,  together 
with  instruments  and  equipment  that  are 
adequate  in  specifications,  checked  for 
calibration  and  maintained  in  good  working 
order. 

Unlike  testing  laboratories,  crime 
laboratories  do  not  generally  have,  nor  are 
required  to  use,  "standard",  "accepted"  or 
"recommended"  methods.    Criminalistics  is  a 
relatively  new  discipline  and  is  comprised 
of  many  identifiable  forensic  science  sub- 
specialties such  as  serology,  firearms 
identification  and  questioned  documents 
identification.    Also,  methods  and  pro- 
cedures are  to  some  degree  dependent  on  the 
requirements  of  the  laws  and  regulations  of 
the  jurisdiction(s)  served  by  the  crime 
laboratory,  and  criminalists  are  concerned 
that  courts  may  discredit  a  scientific 
examination,  however  valid,  if  it  were  not 
carried  out  by  a  "standard"  method. 
Finally,  many  criminalistic  examinations, 
whether  an  identification  of  an  unknown  or 
the  comparison  of  two  or  more  items  to 
determine  common  origin,  are  so  dependent  on 
varying  quantities  of  the  physical  evidence 
available,  or  so  affected  by  the  possibility 
of  contamination,  that  standardization  of 
the  procedure,  method  or  test  is  difficult. 


The  Forensic  Sciences  Foundation  is 
currently  carrying  out  a  LEAA-supported 
project  entitled  "Criminalistics  Methods  of 
Analysis  Feasibility  Study".     Its  purpose  is 
to  develop  and  test  a  mechanism  for  deter- 
mining the  acceptability  of  alternative 
scientific  methods.     Should  a  workable 
mechanism  be  developed,  perhaps  one  that 
would  include  a  compendium  of  acceptable 
scientific  methods,  the  procedures  section 
of  the  ASCLD  accreditation  program  could 
benefit  from  it.     It  should  be  emphasized 
again,  however,  that  the  procedures  or 
methods  used  by  the  crime  laboratory  are  but 
one  aspect  of  the  accreditation  program; 
management  practices,  personnel  qualifi- 
cations, physical  plant,  and  evidence 
security  are  equally  important. 

2.4    Physical  Plant  and  Security 

Space  considerations,  design  consid- 
erations, security  and  personnel  safety  are 
the  elements  of  this  fourth  section  of  the 
accreditation  program.    Adequate  and  proper 
space,  including  bench  areas,  storage  areas, 
and  clerical  and  other  support  space  is  an 
obvious  necessity  for  efficient  operation 
of  any  laboratory.     It  lowers  the  risk  of 
mishandling  or  contaminating  physical 
evidence  and  keeps  personnel  morale  at  a 
level  conducive  to  high  productivity. 
Proper  design  of  the  crime  laboratory  also 
facilitates  its  operation.    For  example, 
inadequacy  or  misplacement  of  utilities  can 
hinder  or  even  prevent  certain  instruments 
from  functioning  properly. 

The  need  for  physical  evidence  security 
and  integrity  sets  a  forensic  science 
laboratory  apart  from  other  types  of 
laboratories.     It  is,  in  fact,  a  serious 
enough  element  within  the  accreditation 
process  that  failure  to  meet  the  standards 
for  "chain  of  evidence"  or  integrity  of 
evidence  would  undoubtedly  cause  a  crime 
laboratory  to  fail. 

To  ensure  that  laboratory  personnel 
work  in  a  safe  environment,  standards  for 
safety  equipment,  for  the  handling  of 
carcinogenic/toxic/dangerous  materials,  and 
safety  procedures  and  rules  must  be 
established  and  adhered  to. 
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3.     Pilot  Program  for  On-Site  Evaluation 
of  Crime  Laboratories 

After  final  review  of  the  Crime 
Laboratory  Accreditation  Program  by  the 
ASCLD  Board  of  Directors,  a  pilot  on-site 
evaluation  test  was  designed  and  completed. 
Four  laboratories  of  varying  size  (from 
four  to  more  than  forty  professional 
personnel),  differing  geographic  location, 
and  organization  placement  ranging  from 
local  to  regional  to  state,  volunteered  to 
serve  as  "test  sites".    The  site  evaluation 
teams  consisted  of  four  (at  one  site,  three) 
crime  laboratory  directors.    The  make-up  of 
the  teams  varied  only  slightly;  many  of  the 
team  members  participated  in  more  than  one 
site  evaluation  as  either  part  of  the 
inspection  team  or  as  a  "host"  laboratory. 

The  report  of  the  on-site  evaluation 
sub-committee  has  not  been  completed. 
Tentative  conclusions  suggest  that  many  of 
the  evaluation  criteria  must  be  re-written 
in  a  form  that  is  more  objective  and  more 
readily  capable  of  measurement.    The  on-site 
visit  protocol  must  be  standardized  in  terms 
of  procedure,  the  time  spent  on  each  section 
of  the  evaluation  criteria,  and  the  training 
of  the  on-site  visit  team  members.  A 
comprehensive  and  well-planned  on-site  visit 
is  an  absolute  necessity  in  a  meaningful 
laboratory  accreditation  program. 

4.     Implementation  of  the  ASCLD 
Accreditation  Program 

Acceptance  of  the  Standards  Manual,  as 
distributed,  by  the  membership  of  the 
American  Society  of  Crime  Laboratory 
Directors  at  its  October,  1979  meeting  will 
signal  the  Laboratory  Evaluation  and 
Standards  Committee  to  request  approval  to 
prepare  a  proposal  for  implementation  of  an 
accreditation  program.    To  be  considered  and 
developed,  then,  would  be  such  factors  as 
1)  the  composition  and  bylaws  of  the 
Laboratory  Accreditation  Board,  2)  the 
length  of  accreditation  period(s),  3)  the 
method  of  application  for  accreditation, 
including  the  types  of  supporting  documents 
that  should  be  submitted  prior  to  the  on- 
site  visit,  4)  the  procedure  for  on-site 
visits,  5)  re-accreditation,  6)  a  "scoring" 
system,  pass/fail  or  otherwise,  and  7) 
financial  support  of  the  program. 


5.     Summary  and  Objectives  of  Accreditation 

Accreditation  of  a  forensic  science 
laboratory  involves  the  assessment  of  a 
number  of  factors,  including  the  management 
of  the  laboratory,  the  general  overall 
competency  of  the  staff,  suitability  of  the 
working  environment  and  the  evaluation  of 
the  procedures  used  in  the  course  of  its 
operation.    A  fundamentally  sound  and 
objective  accreditation  program  will  benefit 
the  criminal  justice  system,  specifically 
the  users  of  crime  laboratory  services,  by 
1)  improving  the  quality  of  laboratory 
services  provided  by  the  criminal  justice 
system,  2)  offering  to  the  general  public 
and  to  users  of  laboratory  services  a  means 
of  identifying  those  laboratory  facilities 
which  satisfy  accreditation  criteria,  3) 
developing  and  maintaining  criteria  which 
can  be  used  by  a  laboratory  to  assess  its 
level  of  performance  and  strengthen  its 
operation,  and  4)  providing  an  independent, 
impartial  and  objective  system  by  which 
laboratory  facilities  can  benefit  from  a 
total  organizational  review. 
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CERTIFIED  LABORATORY  PERFORMANCE  EVALUATION 

H.  JAMES  BARTH 

USDA-FSQS-SCIENCE 
WASHINGTON,  DC  20250 


The  USDA-FSQS-Science  Certified  Laboratory  Program  commenced  in 
late  1962  in  response  to  the  meat  food  industry  for  more  rapid  analyt- 
ical results  on  official  samples.     Initially,  non-government  Chemistry 
laboratories  that  requested  certification  and  that  met  our  require- 
ments were  certified  for  moisture,  protein,  and  salt  determinations. 
In  1969,  with  the  advent  of  the  cooked  sausage  regulations,  the  Certified 
Laboratory  Program  was  extended  to  cover  fat  determinations  in  addition 
to  the  previously  mentioned  determinations. 

Presently,  the  Certified  Laboratory  performance  evaluation  is  an 
ongoing  program.     Analytical  results  are  compared  between  the  Certified 
Laboratories  and  the  USDA  Laboratories.     These  analyses  are  monitored 
using  computer  generated  reports.     Each  USDA  Certified  Laboratory  Reviewer 
is  responsible  for  verifying  these  reports  for  accuracy  and  following  up 
on  unacceptable  analytical  variations.     In  addition,  the  Certified 
Laboratory  Reviewer  makes  an  unannounced  on-site  review  of  each  Certified 
Laboratory  per  year.     Review  reports  contain  information  regarding  all 
aspects  of  the  Certified  Laboratory's  analytical  operation  and  include 
recommendations  for  necessary  corrective  action  including  decertifica- 
tion if  necessary. 

Key  words:     Accuracy  and  precision  of  the  checking  authority;  laboratory 
procedures;  reliability  of  the  sample  collection;  tools  for  laboratory 
evaluation. 


1.  Introduction 

Certified  Laboratory  Performance  evalua- 
tion is  a  multiphase  operation.     In  order  to 
provide  a  fair  evaluation  of  a  laboratory's 
ongoing  performance  a  procedure  had  to  be 
devised  to  consider  three  main  points: 

1.  Accuracy  and  precision  of  the  check- 
ing authority 

2.  The  reliability  of  the  sample 
collector 

3.  The  actual  laboratory  procedures 
both  administrative  and  analytical 

To  ignore  any  one  of  these  points  would 


result  in  a  loss  of  integrity  of  the  overall 
laboratory  evaluation  program. 

2.     Accuracy  And  Precision  Of  The 
Checking  Authority 

The  continued  accuracy  of  the  USDA-FSQS- 
Science  Laboratories  is  monitored  with  known 
check  samples  and  blind  check  samples.  In 
addition,  the  Laboratory  Quality  Control 
Officer  maintains  laboratory  control  charts 
and  requires  that  each  analyst  maintain  his 
own  quality  control  charts.     These  charts 
are  regularly  reviewed  by  the  laboratory 
Quality  Control  Officer  to  determine  that 
work  is  maintained  to  established  performance 
standards . 
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Certified  Laboratory  evaluation  begins 
with  the  sample  collector  who  selects  the 
split  samples  used  to  check  the  certified 
laboratory's  continued  analytical  capability. 
Proper  training  of  the  sample  collector 
includes  sample  selection,  preparation, 
handling,  labeling  and  mailing  procedures. 
Improperly  prepared  samples  may  introduce 
sample  variability,  exceeding  analytical 
variability,  yielding  analytical  results 
whose  variability  could  not  be  blamed  on  the 
laboratory.     We  cannot  realistically  measure 
analytical  variability  under  these  condi- 
tions. 

3.  Laboratory  Procedures 

Laboratory  Quality  Control  is  stressed 
at  all  times,  and  includes  a  careful  critical 
points  evaluation  of  sample  receipt,  record 
keeping,  sample  preparation,  in-laboratory 
sample  tracking,  analutical  procedures,  cal- 
culations, and  reporting  of  results.  Stan- 
dard solutions  and  laboratory  equipment  are 
to  be  regularly  checked  and  records  of  such 
checks  are  to  be  recorded  in  detail  in  the 
Laboratory  Standards  Book.     We  require  the 
laboratory  to  determine  sample  acceptability 
upon  sample  receipt.     Sample  acceptability 
includes  sample  integrity,  proper  identifi- 
cation, and  sufficient  amount  of  sample  for 
the  requested  analysis.     Complete  sample 
information  must  be  kept  in  a  sample  record 
book. 

4.  Tools  For  Laboratory  Evaluation 

The  primary  tool  for  ongoing  laboratory 
performance  evaluation  is  the  companion  sam- 
ple computer  program.     Companion  samples  are 
split  samples  analyzed  by  both  the  Certified 
and  the  assigned  USDA-FSQS  Laboratories. 
Companion  sample  results  are  stored  by  com- 
puter operation  for  each  laboratory.  This 
processed  data  provides  a  profile  of  the  con- 
tinuing laboratory  performance. 

To  keep  data  errors  to  a  minimum,  each 
Certified  Laboratory  Reviewer  is  provided 
with  a  copy  of  the  weekly  summary  of  newly 
matched  data  (between  the  Certified  Labora- 
tory and  the  USDA-FSQS-Science  Laboratory) . 
The  computer  highlighted  data  that  exceeds 
the  accepted  limits  of  analytical  variation 
is  rechecked  to  determine  the  accuracy  of 
the  USDA-FSQS-Science  Laboratory  or  Certified 
Laboratory  data  as  well  as  correctness  of 
data  entered  into  the  data  base.     In  addition, 
the  Certified  Laboratory  Reviewer  contacts 
the  Certified  Laboratory  to  discuss  the 
possibility  of  analytical  problems  occurring 
when  the  computer  report  shows  a  bias 
developing. 


The  Certified  Laboratory  Reviewer  makes 
an  annual  unannounced  on-site  review  of  each 
Certified  Laboratory  and  of  the  Inspectors 
who  utilize  the  Certified  Laboratory.  Review 
reports  contain  information  regarding  all 
aspects  of  the  Certified  Laboratory's  analyt- 
ical and  administrative  operation  and  provide 
recommendations  for  necessary  corrective 
actions  including  possible  decertification. 

The  Certified  Laboratory  coordinator 
maintains  an  overall  laboratory  performance 
file  showing  original  certification  results, 
ongoing  laboratory  performance,  review 
reports,  computer  generated  data  reports, 
and  any  correspondence  regarding  recommenda- 
tions for  corrective  action. 

5.  Summary 

By  constantly  pursuing  any  area  where 
errors  may  occur,  by  continuously  monitoring 
the  check  samples  from  the  Certified  Labora- 
tory and  finally  by  maintaining  a  good  Certi- 
fied Laboratory  Review  Program,  a  good  mea- 
sure of  laboratory  performance  can  be 
achieved . 
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1. 


Introduction 


Task  Force  A  was  to  make  an  analysis 
of  the  legal  problems  raised  by  the 
recognition  of  national  laboratory 
accreditation  systems,  based,  among 
other  things,  upon  the  answers  to  a 
questionnaire  which  was  reviewed  and 
approved  at  ILAC/78. 


The  first  International  Laboratory 
Accreditation  Conference  (ILAC/77),  held 
in  Copenhagen  in  October  1977,  designated 
task  forces  to  explore  relations  with  ISO, 
to  study  legal  problems  associated  with 
Accreditation,  and  to  consider  the  prepar- 
ation of  an  international  registry  of 
accrediting  organizations.  The  second 
Conference  (ILAC/78),  held  in  Washington 
Hn  October  1978,  established  Task  Forces 
jft,  B,  and  C  to  continue  and  extend  the 
vork  of  the  original  task  forces  (i.e., 
Task  Forces  1,  2,  and  3),  and  to  report  on 
:hat  work  at  the  third  Conference 
ILAC/79 )  to  be  held  in  Sydney,  Australia, 
in  October  1979. 


Task  Force  B  was  to  decide  on  informa- 
tion to  be  sought  from  accrediting 
organizations  which  would,  in  turn,  be 
assembled  by  the  Australian  delegate 
into  a  draft  directory  of  organiza- 
tions or  bodies  which  operate  accredi- 
tation systems.  The  Task  Force  would 
analyze  the  problems  encountered  in 
collecting  and  presenting  the  informa- 
tion, including  costs  and  size    of  the 
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task,  and  make  proposals  for  the  main- 
tenance and  dissemination  of  the  directory. 

Task  Force  C  was  to  prepare,  in 
cooperation  with  ISO  and  other  con- 
cerned international  organizations,  a 
paper  on  needs,  objectives,  and 
effects  and  consequences  of  laboratory 
accreditation,  and  to  prepare  a  list 
of  basic  terms  and  their  definitions 
relative  to  laboratory  accreditation. 

These    task    force    reports  constitute 

the  main  items  on  the  agenda  of  the  third 

Conference  (ILAC/79)  to  be  held  in  Sydney 
in  October  1979. 

This  position  paper  is  based  on  the 
reports  of  Task  Forces  A,  B,  and  C.  The 
proposed  draft  directory  of  organizations 
and  the  subsequent  analysis  of  costs, 
size,  maintenance,  and  dissemination  of 
the  directory  have  not  as  yet  been  made 
available  to  ILAC  delegates. 

Comments  on  this  position  paper  and  on 
the  Task  Force  reports  should  be  made  at 
the  preconf erence  briefing  session  to  be 
held  at  2  p.m.  on  September  26,  1979,  at 
the  National  Bureau  of  Standards,  or 
should  be  submitted  in  writing  on  or 
before  October  5,  1979,  to  Dr.  Howard  I. 
Forman,  Deputy  Assistant  Secretary  for 
Product  Standards,  U.S.  Department  of 
Commerce,  Washington,  D.C.  20230;  (202) 
377-3221. 

2.  Overall  U.S.  Position 

The  United  States  supports  the  infor- 
mal assemblage  of  nations  and  interna- 
tional organizations  represented  by  ILAC, 
and  the  objectives  and  work  of  ILAC  as 
reflected  in  the  activities,  and  reports  of 
ILAC's  various  task  forces. 

3.  Report  of  Task  Force  A 
3.1  Summary 

In  response  to  the  questionnaire 
distributed  by  Task  Force  A,  over  600 
pages  of  information  from  18  countries 
were  received  and  analyzed.  Task  Force  A 
considered  not  only  specific  legal  con- 
straints to  the  formal,  national  recogni- 
tion of  accreditation  systems  but  also 
relevant  factors  bearing  on  the  activity 
of  testing  laboratories. 

The  main  points  made  by  Task  Force  A 
in  its  report  are: 


A.  In  many  instances,  mandatory 
requirements  for  testing  products 
prescribe  that  only  specific 
laboratories  named  in  the  law  or 
regulation  may  carry  out  tests 
relevant  to  these  requirements. 
Accordingly,  it  may  be  necessary 
for  such  laws  or  regulations  to  be 
amended  to  allow  other  testing 
laboratories  which  have  been 
accredited  under  a  recognized 
accreditation  program  to  carry  out 
the  appropriate  tests  on  the 
products  covered  by  those  laws  or 
regul at  ions. 

B.  Some  accreditation  bodies  appar- 
ently are  able  to  accredit  foreign 
testing  laboratories;  but  either 
the  extent  of  such  accreditation 
is  not  clear,  or  their  specific 
authority  to  provide  such  accredi- 
tation is  not  clear. 

C.  As  a  general  rule,  countries  do 
not  impose  on  laboratories  in 
foreign  countries  requirements  or 
obligations  which  materially 
differ  from  those  imposed  on  their 
domestic  laboratories. 

D.  There  is  relatively  little  evi- 
dence of  specific  laws  or  regula- 
tions which  immunize  an  accredi- 
ting body  from  liability  or  acts 
of  negligence.  No  court  decisions 
whereby  accrediting  organizations 
or  testing  laboratories  were  found 
liable  for  any  act  of  negligence 
in  accrediting  a  testing  labora- 
tory were  reported. 

E.  Full  reciprocity  for  acceptance  of 
accredited  laboratories  among 
countries  should  be  based  on 
requirements  such  as:  the  need 
for  comparable  standards  and 
procedures;  the  need  for  proced- 
ures to  insure  continuing  product, 
compliance;  the  need  for  the 
accrediting  body  to  be  independent 
of  the  laboratories  it  accredits; 
the  need  for  test  reports  to  be  in 
the  language  of  the  country 
extending  acceptance  of  the 
accreditation  program;  and  the 
need  for  reciprocity  to  be  open 
only  to  countries  extending  mutual 
reciprocity. 


3.2    Recommendations  of  Task  Force  A 

That  Task  Force  A  be  continued  and  be 
assigned  the  following  tasks  (to  be 
reported  at  I  LAC/80  in  Paris): 

A.  Look  further  into  the  existence  of 
legal  reasons  preventing  the 
formal  accreditation  of  foreign 
testing  laboratories.  To  assist 
in  this  task,  request  the  delega- 
tions responding  to  the  Task  Force 
A  questionnaire  to  provide  further 
information  on  the  following  items: 

i.  For  those  accreditation  pro- 
grams which  do  not  permit 
accreditation  of  foreign 
testing  laboratories,  list 
major  reasons  for  refusing  to 
do  so  and  indicate  what  needs 
to  be  done  to  eliminate  such 
restrictions. 

ii.  For  those  accreditation  pro- 
grams which  accredit  foreign 
testing  laboratories,  identify 
any  major  difficulties  which 
may  have  arisen  from  such 
accreditation,  and  list 
accredited  laboratories 
together  with  the  extent  to 
which  such  laboratories  have 
been  used. 

B.  Identify  the  possible  extent  of 
unstated  but  real  problems  of 
those  accreditation  systems  which 
do  not  provide  accreditation  to 
foreign  testing  laboratories. 
Include,  particularly,  problems 
associated  with  the  difficulty  in 
instituting  liability  proceedings 
against  a  foreign  testing  labora- 
tory, and  problems  in  use  of 
public  funds  to  pay  for  such 
accreditation  such  as  cases  where 
the  agency  is  not  willing  to  use 
such  funds  to  accredit  foreign 
testing  laboratories. 

C.  For  those  laws  and  regulations 
silent  on  foreign  laboratory 
accreditation  programs,  ask  the 
reporting  country  to  specify  how 
the  laws  may  be  amended,  revised, 
or  otherwise  interpreted  to  pro- 
vide for  recognition  of  such  pro- 
grams. Task  Force  A  recognizes 
that  appropriate  criteria  for 
evaluating  laboratories  are 
essential  to    the    operation    of  a 


recognized  accreditation  program 
and  that  ILAC  may  wish  to  compile 
suggested  criteria. 

D.  If  ILAC  should  undertake  to  com- 
pile appropriate  criteria,  Task 
Force  A  should  be  invited  to 
examine  those  criteria  to  identify 
any  legal  problems  or  recommend 
what  laws  need  to  be  enacted  or 
amended  to  allow  use  of  such 
criteria. 

E.  Task  Force  A  was  presented  with 
the  possibility  that  mandatory 
testing  systems  may  be  an  impor- 
tant source  of  technical  barriers 
to  trade.  Task  Force  A  should 
determine  the  likelihood  of  this 
possibility  and  propose  ways  and 
means  for  eliminating  or  minimiz- 
ing that  possibility. 

3.3    U.S.  Position 

Concurrence  with  recommendations  A,  B, 
and  C  of  Task  Force  A,  since  these  are 
logical  extensions  of  the  work  already 
begun  and  appear  to  be  necessary  to  more 
completely  utilize  the  data  already  col- 
lected. Concur  also  with  recommendation 
D,  suggesting  that  the  chairman  of  the 
next  scheduled  ILAC  Conference  be  author- 
ized to  request  assistance  from  Task  Force 
A  whenever,  in  his  judgment,  there  is  the 
need  suggested  in  recommendation  D. 

The  U.S.  strenuously  objects  to  recom- 
mendation E  of  Task  Force  A.  ILAC  is  an 
informal  assemblage  of  nations  and  inter- 
national organizations  primarily  inter- 
ested in  developing  arrangements  whereby 
testing  laboratories  and  their  data  in  one 
country  will  be  recognized  and  accepted  as 
valid  in  another  country.  Some  partici- 
pants in  ILAC  may  be  government  officials 
having  responsibility  for,  or  for  other 
reasons  are  concerned  with,  international 
trade  negotiations.  Some  of  those  offi- 
cials have  indicated  a  desire  to  use  ILAC 
as  a  forum  to  discuss  issues  which  are  not 
basic  to  ILAC's  principal  purposes  and 
objectives,  particularly  issues  concerning 
matters  which  have  for  many  years  been  the 
subject  of  discussions  in  the  multilateral 
trade  negotiations  (GATT  Agreement),  and 
which  will  continue  to  be  discussed  in  the 
context  of  the  implementation  of  those 
trade  negotiations. 

The  U.S.  sees  no  logical  reason  why 
ILAC's  forum,  which  is    intended    to    be  a 
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temporary  one  with  plenary  sessions  held  (iv) 
only  once  annually,  should  be  used  to 
discuss  any  of  the  same  issues  that  are, 
by  formal  agreement  of  the  nations 
involved,  subject  to  disposition  by  the  (v) 
terms  of  that  Agreement.  The  lack  of 
logic  in  having  ILAC  spend  any  of  its 
limited  time  on  such  matters  is  even  more 
apparent  in  view  of  the  fact  that  ILAC  has 
no  means  by  which  it  can  resolve  any  such 
matters. 

(vi) 

For  a  more  detailed  explanation  of 
this  position,  reference  is  made  to  a 
letter  of  June  19,  1979,  which  Dr.  Forman, 
Head  of  the  U.S.  Delegation  to  ILAC/79, 
sent  to  the  ILAC/78  Heads  of  Delegations 
from  the  European  Common  Market  Countries. 

4.    Report  of  Task  Force  B 

4.1  Summary 

Task  Force  B  developed  an  outline  for 
the  provisional  directory  shown  in  Annex  1 
of  its  report.  That  outline  proposes  a 
"foreword"  describing  the  background  of 
ILAC,  the  purpose  and  layout  of  the  direc- 
tory, methods  for  collecting  information, 
and  methods  for  collecting  updated  infor- 
mation in  the  directory.  The  outline  pro- 
poses "Part  I"  of  the  directory  to  include 
one-page  summaries  of  the  general  testing 
arrangements  in  each  of  the  countries 
represented  in  ILAC.  The  outline  proposes 
"Part  II"  of  the  directory  which  would 
contain  one-page  summaries  of  each  of  the 
accreditation  programs  existing  in  each  of 
the  countries  represented  at  ILAC. 

To  be  eligible  for  inclusion  in 
Part  II  of  this  directory  an  accrediting 
organization  must  operate  its  accredita- 
tion system  to  meet  the  following  condi- 
tions: 

(i)    It  must  be  actively  engaged  in  the 
accreditation  of  laboratories. 

(ii)  It  must  specify  accreditation  of 
the  testing  laboratories  in  terms 
of  well-defined  fields  of  testing, 
scientific  disciplines  or  tech- 
nologies, or  specific  products  or 
tests. 

(iii)  Its  technical  criteria  for 
accreditation  must  be  formulated 
by  persons  possessing  the  neces- 
sary technical  competence  in  the 
relevant  field  of  testing.  (vii) 


Its  criteria  for  accreditation 
must  be  published  and  be  generally 
avail  able. 

It  must  only  accredit  laboratories 
in  respect  of  tests  performed  in 
accordance  with  test  specifica- 
tions agreed  between  the  labora- 
tory and  the  accrediting  organi- 
zation. 

Its  evaluation  of  laboratories 
must  be  in  terms  of  criteria  which 
provide  that: 

(a)  Laboratory  structure  and 
management  are  clearly  defined 
and  are  organized  in  such  a 
way  that  the  integrity  of  its 
staff  and  operation  is  assured; 

(b)  Laboratory  staff    are  suitably 
qualified  and    experienced  and 
technically  competent    for    the   j  j 
work  in  which  they  are  engaged;    •  n 

jj  6 

(c)  Laboratory  equipment  is  appro- 
priate   for    the    tests  under- 
taken and  that  it    is  properly 
installed,     maintained,  and 
calibrated    at    intervals  pre- 
scribed   by     the     accrediting  \  tj 
authority.      Adequate     records  ;8 
of    calibration    and    servicing  t  j 
must  be  maintained; 

'  in 

(d)  The  testing  environment  and  j  j 
laboratory  accommodation  are  r  tj 
appropriate  for  the  tests  c  fa 
undertaken; 

'  ill 

(e)  Laboratory  practices  such  as--    i  k 

sampling  (where  appropriate) 
sampling  identification 
test  methods  and  procedures 
supervision  of  staff 
keeping  of  records 
checking  of  results  and  cal-  , 
culations  are  satisfactory;  r  \\ 

i  (j 

(f)  The  laboratory    records  system 

is  secure  and  contains  full  r,  ;jt( 
details  of  all  tests  under-  :  ^ 
taken; 

k 

(g)  Test  documents    present    accur-  I  tj), 
ately,  clearly,    and    unambigu-  ;  jrj 
ously  the  test  results  and  all 
relevant  information. 

(No  item  vii  was    included    in  the 
Report. ) 
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(viii)  It  must  assess  laboratories  using 
teams  of  impartial  experts  who 
have  expertise  in  the  area  of 
testing  in  which  accreditation  is 
sought. 

(ix)  Its  assessment  must  be  completed 
by  a  formal  report  by  the 
assessors. 

(x)  It  must  reassess  its  accredited 
laboratories  at  regular  and 
reasonable  intervals. 

(xi)  It  must  publish  a  list  of  labora- 
tories which  it  has  accredited. 
Entries  in  Part  II  are  to  be  made 
in  terms  of  nine  fields  of  testing 
(chemical,  electrical,  etc.). 

4.2  Recommendations  of  Task  Force  B 

Task  Force  B  agreed  to  meet  in  Sydney 
immediately  prior  to  ILAC/79  to  update  its 
report,  and  to  meet  again  after  ILAC/79  to 
expedite  the  work  and  take  into  account 
the  comments  made  at  ILAC/79. 

4.3  U.S.  Position 

Task  Force  B  should  be  encouraged  to 
continue  the  development  of  the  direc- 
tory. A  plan  for  developing  and  dis- 
tributing the  directory  in  final  form 
should  be  prepared  by  the  Task  Force  for 
presentation  at  ILAC/80.  The  draft  direc- 
tory should  be  distributed  to  all  ILAC 
representatives  as  soon  as  possible  for 
comments.  The  task  force  should  resolve 
all  the  issues  raised  by  the  comments  and, 
alternatives  for  presentation  at  ILAC/80 
for  resolution. 

The  "Conditions  for  Entry  in  Part  II 
of  Directory"  presented  in  Annex  III  of 
the  Task  Force  report  and  reprinted  above 
represent  criteria  for  evaluating  accredi- 
tation organizations.  Although  these  con- 
ditions appear  to  be  reasonable,  there  are 
many  which  will  be  subject  to  interpreta- 
tion (e.g.,  "(vi )(e) ...  laboratory  prac- 
tices such  as  sampl i ng. . . are  satisfac- 
tory."). The  U.S.  believes  that  ILAC/79 
or  ILAC/80  should  vote  on  each  condition 
listed  in  Annex  III  before  these  condi- 
tions are  put  into  effect  in  the  final 
directory. 


5.    Report  of  Task  Force  C 

5.1  Summary 

Task  Force  C  presented  a  report  on  the 
needs,  objectives,  and  effects  and  con- 
sequences of  laboratory  accreditation 
(Appendix  1  of  the  Task  Force  C  report). 
The  intent  of  the  report  was  to  discuss 
why  laboratory  accreditation  is  desirable, 
what  it  can  hope  to  achieve,  and  the 
impact  such  accreditation  would  have  on 
the  community  which  utilizes  such  a  system. 

Appendix  2  to  the  report  includes  a 
set  of  definitions  of  basic  terms  used  in 
laboratory  accreditation.  Definitions 
compatible  with  ISO  definitions  were  used 
wherever  possible. 

5.2  Recommendations  of  Task  Force  C 

The  Task  Force  submitted  the  following 
recommendations: 

A.  ILAC/79  should  encourage  all  na- 
tions to  develop  national  systems 
for  accreditation  of  their  testing 
laboratories  along  lines  which 
general ly  wi 1 1  be  recognized  and 
utilized  by  other  nations. 

B.  Work  begun  in  providing  the  defin- 
itions shown  in  Appendix  II  of  the 
Task  Force  report  should  be  con- 
tinued in  an  attempt  to  arrive  at 
a  comprehensive  set  of  defini- 
tions. ISO  should  be  invited  to 
present  such  a  set  of  definitions 
to  ILAC/80,  but  if  ISO  does  not 
accept  this  invitation  by  Janu- 
ary 1,  1980,  ILAC  should  request 
that  Task  Force  C  continue  the 
work. 

C.  A  thorough  comparison  of  the  cri- 
teria used  by  the  major  accredita- 
tion systems  should  be  undertaken 
by  Task  Force  C  for  ILAC/80  lead- 
ing to  minimum  criteria  for  multi- 
lateral recognition. 

D.  Each  country  should  establish  a 
central  inquiry  point  to  provide 
information  on  accreditation 
systems  and  accredited  labora- 
tories. 
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5.3    U.S.  Position 


Suggest  that  the  report  and  Appendix  1 
be  incorporated  as  the  introduction  to  the 
proposed  directory  and  that  the  defini- 
tions, Appendix  II,  be  included  in  the 
appropriate  section  of  the  directory. 
Further  suggest  that  Task  Force  B  be  given 
the  responsibility  for  receiving  suggested 
revisions  to  the  definitions  and  for  pre- 
paring a  set  of  final  definitions  for  use 
in  the  directory. 
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JOURNAL  OF  RESEARCH— The  Journal  of  Research  of  the 
National  Bureau  of  Standards  reports  NBS  research  and  develop- 
ment in  those  disciplines  of  the  physical  and  engineering  sciences  in 
which  the  Bureau  is  active.  These  include  physics,  chemistry, 
engineering,  mathematics,  and  computer  sciences.  Papers  cover  a 
broad  range  of  subjects,  with  major  emphasis  on  measurement 
methodology  and  the  basic  technology  underlying  standardization. 
Also  included  from  time  to  time  are  survey  articles  on  topics 
closely  related  to  the  Bureau's  technical  and  scientific  programs. 
As  a  special  service  to  subscribers  each  issue  contains  complete 
citations  to  all  recent  Bureau  publications  in  both  NBS  and  non- 
NBS  media.  Issued  six  times  a  year.  Annual  subscription:  domestic 
$13;  foreign  $16.25.  Single  copy,  $3  domestic;  $3.75  foreign. 
NOTE:  The  Journal  was  formerly  published  in  two  sections:  Sec- 
tion A  "Physics  and  Chemistry"  and  Section  B  "Mathematical 
Sciences." 

DIMENSIONS/NBS — This  monthly  magazine  is  published  to  in- 
form scientists,  engineers,  business  and  industry  leaders,  teachers, 
students,  and  consumers  of  the  latest  advances  in  science  and 
technology,  with  primary  emphasis  on  work  at  NBS.  The  magazine 
highlights  and  reviews  such  issues  as  energy  research,  fire  protec- 
tion, building  technology,  metric  conversion,  pollution  abatement, 
health  and  safety,  and  consumer  product  performance.  In  addi- 
tion, it  reports  the  results  of  Bureau  programs  in  measurement 
standards  and  techniques,  properties  of  matter  and  materials, 
engineering  standards  and  services,  instrumentation,  and 
automatic  data  processing.  Annual  subscription:  domestic  $11; 
foreign  $13.75. 

NONPERIODICALS 

Monographs — Major  contributions  to  the  technical  literature  on 
various  subjects  related  to  the  Bureau's  scientific  and  technical  ac- 
tivities. 

Handbooks — Recommended  codes  of  engineering  and  industrial 
practice  (including  safety  codes)  developed  in  cooperation  with  in- 
I    terested  industries,  professional  organizations,  and  regulatory 
bodies. 

Special  Publications — Include  proceedings  of  conferences  spon- 
sored by  NBS,  NBS  annual  reports,  and  other  special  publications 
appropriate  to  this  grouping  such  as  wall  charts,  pocket  cards,  and 
bibliographies. 

Applied  Mathematics  Series — Mathematical  tables,  manuals,  and 
studies  of  special  interest  to  physicists,  engineers,  chemists, 
biologists,  mathematicians,  computer  programmers,  and  others 
engaged  in  scientific  and  technical  work. 

National  Standard  Reference  Data  Series — Provides  quantitative 
data  on  the  physical  and  chemical  properties  of  materials,  com- 
piled from  the  world's  literature  and  critically  evaluated. 
Developed  under  a  worldwide  program  coordinated  by  NBS  under 
the  authority  of  the  National  Standard  Data  Act  (Public  Law 
90-396). 
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published  quarterly  for  NBS  by  the  American  Chemical  Society 
(ACS)  and  the  American  Institute  of  Physics  (AIP).  Subscriptions, 
reprints,  and  supplements  available  from  ACS,  1 155  Sixteenth  St., 
NW,  Washington,  DC  20056. 

Building  Science  Series — Disseminates  technical  information 
developed  at  the  Bureau  on  building  materials,  components, 
systems,  and  whole  structures.  The  series  presents  research  results, 
test  methods,  and  performance  criteria  related  to  the  structural  and 
environmental  functions  and  the  durability  and  safety  charac- 
teristics of  building  elements  and  systems. 

Technical  Notes — Studies  or  reports  which  are  complete  in  them- 
selves but  restrictive  in  their  treatment  of  a  subject.  Analogous  to 
monographs  but  not  so  comprehensive  in  scope  or  definitive  in 
treatment  of  the  subject  area.  Often  serve  as  a  vehicle  for  final 
reports  of  work  performed  at  NBS  under  the  sponsorship  of  other 
government  agencies. 

Voluntary  Product  Standards — Developed  under  procedures 
published  by  the  Department  of  Commerce  in  Part  10,  Title  15,  of 
the  Code  of  Federal  Regulations.  The  standards  establish 
nationally  recognized  requirements  for  products,  and  provide  all 
concerned  interests  with  a  basis  for  common  understanding  of  the 
characteristics  of  the  products.  NBS  administers  this  program  as  a 
supplement  to  the  activities  of  the  private  sector  standardizing 
organizations. 

Consumer  Information  Series — Practical  information,  based  on 
NBS  research  and  experience,  covering  areas  of  interest  to  the  con- 
sumer. Easily  understandable  language  and  illustrations  provide 
useful  background  knowledge  for  shopping  in  today's  tech- 
nological marketplace. 
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ments, Government  Printing  Office,  Washington,  DC  20402. 
Order  the  following  NBS  publications — FIPS  and  NBSIR's — from 
the  National  Technical  Information  Services,  Springfield,  VA  22161 . 

Federal  Information  Processing  Standards  Publications  (FIPS 
PUB) — Publications  in  this  series  collectively  constitute  the 
Federal  Information  Processing  Standards  Register.  The  Register 
serves  as  the  official  source  of  information  in  the  Federal  Govern- 
ment regarding  standards  issued  by  NBS  pursuant  to  the  Federal 
Property  and  Administrative  Services  Act  of  1949  as  amended. 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented  by  Ex- 
ecutive Order  1 1 7 1 7  (38  FR  1 23 1 5,  dated  May  1 1 ,  1 973)  and  Part  6 
of  Title  15  CFR  (Code  of  Federal  Regulations). 

NBS  Interagency  Reports  (NBSIR) — A  special  series  of  interim  or 
final  reports  on  work  performed  by  NBS  for  outside  sponsors 
(both  government  and  non-government).  In  general,  initial  dis- 
tribution is  handled  by  the  sponsor;  public  distribution  is  by  the 
National  Technical  Information  Services,  Springfield,  VA  22161, 
in  paper  copy  or  microfiche  form. 
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Liquefied  Natural  Gas.  A  literature  survey  issued  quarterly.  Annual 
subscription:  $20. 


Superconducting  Devices  and  Materials.  A  literature  survey  issued 
quarterly.  Annual  subscription:  $30.  Please  send  subscription  or- 
ders and  remittances  for  the  preceding  bibliographic  services  to  the 
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